JP7014930B1 - Earth bucket recognition system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an earth bucket recognition system and a program for accurately recognizing the posture of an earth bucket. SOLUTION: In an earth bucket recognition system that recognizes the posture of an earth bucket in a cason work room, an acquisition means for acquiring point cloud data indicating distance information from a position of a sensor in the work room to an acquisition target. An extraction means for extracting the point cloud data of the earth bucket in the work room of the cason from the point cloud data in the work room of the cason acquired by the acquisition means, and a point cloud of the earth bucket extracted by the extraction means. A conversion means for converting the data into coordinate data in the three-dimensional space, a mathematical model consisting of a set of coordinates satisfying the equation indicating the side surface of the cylinder in the three-dimensional space, and the coordinate data converted by the conversion means. It is characterized by providing a recognition means for recognizing the posture of the earth bucket based on an error when the above is superimposed. [Selection diagram] FIG. 4

Description

The present invention relates to an earth bucket recognition system and a program for recognizing an earth bucket in a caisson work room.

The pneumatic caisson method is known as a method for constructing underground structures such as bridges, foundations of buildings, and shafts for shield tunnels. In the pneumatic caisson method, a work room is provided in the lower part of the main body of the caisson, and compressed air is sent into the working room to create a high pressure state, and excavation work is performed. Since this high pressure work room is in a high pressure state, the time that workers can enter is limited. For this reason, in the pneumatic caisson construction method, unmanned construction is adopted in which construction is performed by remote control from the ground from the viewpoint of improving work efficiency and safety of the working environment in construction underwater or under high pressure. In such unmanned construction, an image of the working condition in the high-pressure work room taken by a surveillance camera installed in the high-pressure work room was installed in a safe remote work room on land away from the high-pressure work room. It is displayed on the monitor, and the operator remotely controls the work machine such as an excavator while looking at this screen to perform the construction.

In addition, a remote construction management system that extracts the display target in the work room from the image of the work situation taken by the surveillance camera is disclosed so that even a remote worker can easily grasp the realization site environment. (For example, see Patent Document 1).

According to the disclosed technology of Patent Document 1, a caisson work room is imaged, a display device is used outside the work room, the user can wear the image, and the captured image is displayed, and the display control unit is used by the user. The position of the area to be displayed in the work room is changed according to the posture and displayed on the display device. This makes it possible for the operator to work from a safe remote control room as if he were at the realization site.

Japanese Unexamined Patent Publication No. 2019-71592

In the above-mentioned pneumatic caisson method, the earth and sand excavated by the excavator in the caisson work room are piled up in one place in the caisson work room. The pile of earth and sand is packed using an excavator into an earth bucket suspended by a wire from the ground to transport the earth and sand. The earth bucket filled with earth and sand by the excavator is pulled up to the ground, and the earth and sand packed in the earth bucket is transported to the ground. Through these series of operations, the earth and sand excavated by the excavator in the caisson work room can be transported to the ground.

By the way, the posture of the earth bucket in the caisson's work room changes greatly depending on, for example, the inclination and shape of the ground on which the earth bucket is placed. For example, if the earth bucket is placed on a sloping ground, the earth bucket will tilt according to the slope of the ground. In order to pack earth and sand into the tilted ground bucket, it is necessary to adjust the position and angle of the excavator according to the tilt of the ground bucket and remotely control the excavator. That is, when the work of stuffing the earth and sand into the earth bucket is remotely controlled by using the excavator, it is required to operate the excavator with high accuracy according to the posture of the earth bucket. Therefore, it is necessary to accurately recognize the attitude of the earth bucket.

On the other hand, Patent Document 1 does not describe a method for accurately estimating the angle of inclination of the earth bucket and the like. Therefore, the disclosed technique of Patent Document 1 has a problem that the posture of the earth bucket cannot be accurately recognized.

Therefore, the present invention has been devised in view of the above-mentioned problems, and an object of the present invention is to provide an earth bucket recognition system and a program for accurately recognizing the posture of the earth bucket. ..

The earth bucket recognition system according to the first invention is the earth bucket recognition system that recognizes the posture of the earth bucket in the work room of Kason, and is point group data showing the distance information from the position of the sensor in the work room to the acquisition target. And the extraction means for extracting the point group data of the earth bucket in the work room of the casen from the point group data in the work room of the cason acquired by the acquisition means, and the extraction means. A conversion means for converting the point group data of the ground bucket to coordinate data in the three-dimensional space, a mathematical model consisting of a set of coordinates satisfying an equation indicating the side surface of the cylinder in the three-dimensional space, and the conversion means. It is characterized by comprising a recognition means for recognizing the posture of the earth bucket based on an error when superimposing the coordinate data converted by the above.

In the first invention, the earth bucket recognition system according to the second invention uses the mathematical model in three dimensions so that the recognition means has a smaller error when the mathematical model and the coordinate data are superimposed. It is characterized in that the correction is performed by rotating the coordinate axis in space around the center, and the error when the corrected mathematical model and the coordinate data are superimposed is repeatedly calculated.

The earth bucket recognition system according to the third invention has the coordinates in the first invention or the second invention so that the recognition means has a smaller error when the mathematical model and the coordinate data are superimposed. The data is corrected by rotating it around a coordinate axis in a three-dimensional space, and the error when the mathematical model and the corrected coordinate data are superimposed is repeatedly calculated.

The earth bucket recognition system according to the fourth invention is the second invention or the third invention, wherein the recognition means is the mathematical model or the mathematical model when the error when the mathematical model and the coordinate data are superimposed is minimized. It is characterized in that the posture of the earth bucket is recognized based on an angle rotated about a coordinate axis in a three-dimensional space in order to correct the coordinate data.

In any one of the first to fourth inventions, the earth bucket recognition system according to the fifth invention has a smaller error when the recognition means superimposes the mathematical model and the coordinate data. As described above, the mathematical model is translated in a three-dimensional space and corrected, and the error when the corrected mathematical model and the coordinate data are superimposed is repeatedly calculated.

In any one of the first to fifth inventions, the earth bucket recognition system according to the sixth invention has a smaller error when the recognition means superimposes the mathematical model and the coordinate data. As described above, the coordinate data is translated in a three-dimensional space and corrected, and the error when the mathematical model and the corrected coordinate data are superimposed is repeatedly calculated.

The earth bucket recognition system according to the seventh invention, in the fifth invention or the sixth invention, the recognition means is the mathematical model or the mathematical model when the error when the mathematical model and the coordinate data are superimposed is minimized. It is characterized in that the posture of the earth bucket is recognized based on the position translated in the three-dimensional space in order to correct the coordinate data.

In any one of the first to seventh inventions, the earth bucket recognition system according to the eighth invention acquires the point cloud data for extraction in the work room from which the earth bucket has been removed in advance by the acquisition means. The extraction means extracts the point cloud data of the earth bucket by deleting the point cloud data for extraction acquired by the acquisition means from the point cloud data in the work room where the earth bucket is located. It is a feature.

In any one of the first to eighth inventions, the earth bucket recognition system according to the ninth invention has the acquisition means imaged in the working room of the cason, and the extraction means is acquired by the acquisition means. The point cloud data of the earth bucket is obtained by referring to the color of the earth bucket and extracting the point cloud data corresponding to the position of the color in the image captured by the acquisition means from the point cloud data. It is characterized by extracting.

The earth bucket recognition program according to the tenth invention is a point cloud data indicating distance information from the position of a sensor in the work room to an acquisition target in the earth bucket recognition program that recognizes the posture of the earth bucket in the work room of Kason. And the extraction step of extracting the point cloud data of the earth bucket in the work room of the cason from the point cloud data of the cason's work room acquired by the acquisition step, and the extraction step of the extraction step. A conversion step for converting the point cloud data of the earth bucket into coordinate data in the three-dimensional space, a mathematical model consisting of a set of coordinates satisfying an equation indicating the side surface of the cylinder in the three-dimensional space, and the conversion step. It is characterized in that a computer is made to perform a recognition step of recognizing the posture of the earth bucket based on an error when superimposing the coordinate data converted by the above.

According to the first to ninth inventions, the earth bucket recognition system of the present invention is when a mathematical model consisting of a set of coordinates satisfying an equation indicating the side surface of a cylinder in three-dimensional space and coordinate data are superimposed. Recognize the posture of the earth bucket based on the error of. This makes it possible to recognize the position and inclination of the earth bucket, and when using an excavator to remotely control the work of stuffing earth and sand into the earth bucket, operate the excavator accurately according to the posture of the earth bucket. can do.

In particular, according to the second invention, the earth bucket recognition system of the present invention has a coordinate axis in three-dimensional space with respect to the axial direction of the cylinder so that the error when the mathematical model and the coordinate data are superimposed is reduced. The angle of is corrected, and the error when the corrected mathematical model and the coordinate data are superimposed is calculated. This makes it possible to calculate an error with a mathematical model that is closer to the attitude of the earth bucket, and it is possible to recognize the attitude of the earth bucket more accurately.

In particular, according to the third invention, the earth bucket recognition system of the present invention has a coordinate axis in three-dimensional space with respect to the axial direction of the cylinder so that the error when the mathematical model and the coordinate data are superimposed is reduced. The angle of is corrected, and the error when the mathematical model and the corrected coordinate data are superimposed is calculated. This makes it possible to calculate an error with the coordinate data closer to the attitude of the earth bucket, and it is possible to recognize the attitude of the earth bucket more accurately.

In particular, according to the fourth invention, the earth bucket recognition system of the present invention has a coordinate axis in three-dimensional space with respect to the axial direction of the cylinder when the error when the mathematical model and the coordinate data are superimposed is minimized. Recognize the posture of the earth bucket based on the angle. This makes it possible to recognize the attitude of the earth bucket more accurately, and when using an excavator to remotely control the work of stuffing earth and sand into the earth bucket, the excavator will be more accurate according to the attitude of the earth bucket. It can be operated well.

In particular, according to the fifth invention, the earth bucket recognition system of the present invention translates and corrects so that the error when the mathematical model and the coordinate data are superimposed is small, and the corrected mathematical model and the coordinates. Calculate the error when superimposing on the data. This makes it possible to calculate an error with a mathematical model that is closer to the attitude of the earth bucket, and it is possible to recognize the attitude of the earth bucket more accurately.

In particular, according to the sixth invention, the earth bucket recognition system of the present invention translates and corrects in a three-dimensional space so that the error when the mathematical model and the coordinate data are superimposed is small, and mathematics. Calculate the error when the model and the corrected coordinate data are superimposed. This makes it possible to calculate an error with the coordinate data closer to the attitude of the earth bucket, and it is possible to recognize the attitude of the earth bucket more accurately.

In particular, according to the seventh invention, the earth bucket recognition system of the present invention is based on the translated position in the three-dimensional space when the error when the mathematical model and the coordinate data are superimposed is minimized. Recognize the posture of the earth bucket. This makes it possible to recognize the attitude of the earth bucket more accurately, and when using an excavator to remotely control the work of stuffing earth and sand into the earth bucket, the excavator will be more accurate according to the attitude of the earth bucket. It can be operated well.

In particular, according to the eighth invention, the earth bucket recognition system of the present invention extracts the point cloud data of the earth bucket by deleting the point cloud data for extraction from the point cloud data in the work room where the earth bucket is located. .. This makes it possible to accurately extract the point cloud data of the earth bucket from the two types of point cloud data, and it is possible to recognize the attitude of the earth bucket more accurately.

In particular, according to the ninth invention, the earth bucket recognition system of the present invention refers to the color of the earth bucket from the point cloud data and extracts the point cloud data according to the position of the color in the image to make the earth. Extract the point cloud data of the bucket. This makes it possible to accurately extract the point cloud data of the earth bucket from the image and the point cloud data, and it is possible to recognize the attitude of the earth bucket more accurately.

According to the tenth invention, the earth bucket recognition program of the present invention is based on an error when a mathematical model consisting of a set of coordinates satisfying an equation indicating a side surface of a cylinder in three-dimensional space and coordinate data are superimposed. And recognize the posture of the earth bucket. This makes it possible to recognize the position and inclination of the earth bucket, and when using an excavator to remotely control the work of stuffing earth and sand into the earth bucket, operate the excavator accurately according to the posture of the earth bucket. can do.

FIG. 1 is a vertical sectional view showing the main equipment of the pneumatic caisson method. FIG. 2 is a side view of an excavator which is an example of a working machine according to the present invention. FIG. 3 is a block diagram showing a control system in an excavator. FIG. 4 is a block diagram showing an overall configuration of an earth bucket recognition system to which the invention is applied. FIG. 5 is a flowchart of the operation of the earth bucket recognition system to which the present invention is applied. FIG. 6 is a diagram showing an example of a method for extracting point cloud data of an earth bucket using point cloud data for extraction. FIG. 7 is a diagram showing an example of a method of extracting point cloud data of an earth bucket using an image and point cloud data. FIG. 8 is a diagram showing an example of a mathematical model. FIG. 9 is a diagram showing an example of a procedure for recognizing the posture of the earth bucket using a mathematical model and coordinate data.

FIG. 1 is a diagram showing an example of a main facility of a pneumatic caisson method in which an excavator, which is an example of a working machine according to the present invention, is used. In the pneumatic caisson method, the reinforced concrete caisson 1 is submerged underground using the excavation equipment E1, the equipment equipment E2, the soil removal equipment E3, the air supply equipment E4, and the reserve / safety equipment E5. It is configured to build a structure.

The excavation equipment E1 includes, for example, an excavator 100 (hereinafter referred to as a caisson excavator 100), an automatic earth and sand loading device 11, and a ground remote control room 13. The caisson excavator 100 is installed in the work room 2 provided at the bottom of the caisson 1. The earth and sand automatic loading device 11 loads the earth and sand excavated by the caisson excavator 100 into the cylindrical earth bucket 31. The ground remote control room 13 includes a remote control device 12 that remotely controls the operation of the caisson excavator 100 from the ground.

The fitting equipment E2 includes, for example, a man shaft 21, a man lock 22 (airlock), a material shaft 23, and a material lock 24 (airlock). The man shaft 21 is a cylindrical passage connecting the ground and the work room 2 for an operator to enter and exit the work room 2, and is provided with, for example, a spiral staircase 25. The man lock 22 is an airtight door having a double door structure provided on the man shaft 21 to adjust the atmospheric pressure on the ground and the pressure difference in the work room 2. The material shaft 23 is a cylindrical passage connecting the ground and the work room 2 in order to carry the earth bucket 31 on which the earth and sand are loaded by the earth and sand automatic loading device 11 to the ground. The material lock 24 is an airtight door having a double door structure that adjusts the atmospheric pressure on the ground and the pressure difference in the work room 2 provided on the material shaft 23 and the material shaft 23 for carrying in and out materials. The man lock 22 and the material lock 24 are configured so that the worker and the earth bucket 31 can move in and out of the work room 2 by suppressing the change in the air pressure in the work room 2.

The soil removal equipment E3 includes, for example, an earth bucket 31, a carrier device 32, and a soil hopper 33. The earth bucket 31 is a bottomed cylindrical tubular container on which earth and sand excavated by the caisson excavator 100 are loaded. The carrier device 32 is a device that pulls the earth bucket 31 up to the ground via the material shaft 23 and carries it out. The earth and sand hopper 33 is a facility for temporarily storing the earth and sand carried to the ground by the earth bucket 31 and the carrier device 32.

The air supply equipment E4 includes, for example, an air compressor 42, an air purifier 43, an air supply pressure adjusting device 44, and an automatic depressurizing device 45. The air compressor 42 is a device that sends compressed air into the working room 2 through the air supply passage 3 formed in the air supply pipe 41 and the caisson 1. The air purifying device 43 is a device that purifies the compressed air sent by the air compressor 42. The air supply pressure adjusting device 44 is a device that adjusts the amount (pressure) of the compressed air sent from the air compressor 42 into the working room 2 so that the air pressure in the working room 2 becomes equal to the groundwater pressure. The automatic decompression device 45 is a device that depressurizes the air pressure in the man lock 22.

The spare / safety equipment E5 includes, for example, an emergency air compressor 51 and a hospital lock 53. The emergency air compressor 51 is a device capable of sending compressed air into the work room 2 in place of the air compressor 42 when the air compressor 42 fails or is inspected. The hospital lock 53 is a decompression chamber for a worker who has worked in the work room 2 to enter and gradually acclimatize the worker's body to atmospheric pressure.

Next, the caisson excavator 100 according to the present invention will be described with reference to FIGS. 2 to 3. As shown in FIG. 2, the caisson excavator 100 includes, for example, a traveling body 110, a boom 130, and a bucket attachment 150. The traveling body 110 is attached to a pair of left and right traveling rails 4 provided on the ceiling of the work room 2, and travels along the traveling rails 4 while being suspended from the left and right traveling rails 4. The boom 130 is pivotally connected to the turning frame 121 of the traveling body 110 so as to be swingable in the vertical direction. The bucket attachment 150 is attached to the tip of the boom 130.

The traveling body 110 includes a traveling frame 111, a turning frame 121, and a traveling roller 113. The swivel frame 121 is provided on the lower surface side of the traveling frame 111 so as to be swivelable. The traveling roller 113 is four rollers on the front, rear, left, and right provided on the front and rear on the upper surface side of the traveling frame 111. The traveling body 110 is configured to rotate and drive the traveling rollers 113 in the front-rear and left-right directions to travel along the left and right traveling rails 4.

The boom 130 includes, for example, a base end boom 131, a tip end boom 132, a telescopic cylinder 133, and an undulating cylinder 134. The base end boom 131 is attached to the swivel frame 121 so as to be undulating or swingable in the vertical direction. The tip boom 132 is nested and configured with the proximal boom 131. The telescopic cylinder 133 is provided in the base end boom 131. Two undulating cylinders 134 are provided on the left and right sides of the base end boom 131. The boom 130 is configured such that when the telescopic cylinder 133 is expanded and contracted, the tip boom 132 moves in the longitudinal direction with respect to the proximal boom 131, whereby the boom 130 expands and contracts. The base ends of the two undulating cylinders 134 are rotatably attached to the left and right sides of the base end boom 131, respectively.

The bucket attachment 150 includes a base member 151, a bucket 152, and a bucket cylinder 153. The base member 151 is attached to the tip boom 132. The bucket 152 is attached to the tip of the base member 151 so as to be vertically swingable. The bucket cylinder 153 is configured to swing the bucket 152 up and down with respect to the base member 151.

As shown in FIG. 3, the control unit 165 includes a main controller 165a, a traveling body controller 165b, and a boom / bucket controller 165c. Further, the control unit 165 may be connected to the caisson excavator 100 and the remote control device 12. The control unit 165 may be built in the remote control device 12. The main controller 165a is connected to the traveling body controller 165b and the boom / bucket controller 165c, receives an operation signal from the remote control device 12, and outputs a drive control signal corresponding to the operation signal to the traveling body controller 165b. And output to the boom bucket controller 165c. The vehicle controller 165b is configured to drive the vehicle 110 in response to a drive control signal output from the main controller 165a. The main controller 165a and the traveling body controller 165b are arranged on the turning frame 121 of the traveling body 110. The boom / bucket controller 165c is configured to drive the boom 130 and the bucket attachment 150 in response to the drive control signal output from the main controller 165a. The boom bucket controller 165c is arranged on the side of the base end boom 131 of the boom 130.

As shown in FIG. 3, the cason excavator 100 includes a traveling body position sensor 201, a turning angle sensor 202, a boom undulation angle sensor 203, a boom extension amount sensor 204, a bucket swing angle sensor 205, and an outside world sensor 206. And prepare. The traveling body position sensor 201 detects the position of the traveling body 110 on the traveling rail 4. The turning angle sensor 202 detects the turning angle of the turning frame 121 with respect to the traveling frame 111. The boom undulation angle sensor 203 detects the undulation angle of the boom 130 with respect to the swivel frame 121. The boom extension amount sensor 204 detects the extension amount of the boom 130. The bucket swing angle sensor 205 detects the swing angle of the bucket 152 with respect to the base member 151 of the boom 130 or the bucket attachment 150. The outside world sensor 206 is provided on the traveling body 110 and acquires information such as the distance to the excavated ground in the work room 2 and the shape of the ground. Further, the caisson excavator 100 may communicate with the remote control device 12 and the control unit 165, and transmit the data obtained by the sensors 201 to 206 to the remote control device 12 and the control unit 165.

The traveling body position sensor 201 is composed of, for example, a laser sensor arranged on the traveling frame 111 of the traveling body 110. The traveling body position sensor 201 irradiates the laser beam toward the end of the traveling rail 4 or the wall of the working room 2 until it is reflected at the end of the traveling rail 4 or the wall of the working room 2 and returns. Measure the time of. The traveling body position sensor 201 detects the distance from the end portion of the traveling rail 4 or the wall portion of the work room 2 to the traveling body 110 based on this time. The turning angle sensor 202 is composed of, for example, an optical rotary encoder arranged on a turning frame 121 of the traveling body 110. The turning angle sensor 202 converts the turning amount of the turning frame 121 with respect to the traveling frame 111 into an electric signal. The turning angle sensor 202 calculates and processes the signal to detect a turning angle including the turning direction and position of the turning frame 121. The traveling body position sensor 201 and the turning angle sensor 202 have been described as an example, and other sensors for detecting the two-dimensional position of the traveling body and other sensors for detecting the turning angle of the turning frame 121 are used, respectively. You may.

The boom undulation angle sensor 203 is composed of, for example, a laser sensor arranged on the side of the cylinder bottom of the undulation cylinder 134. The boom undulation angle sensor 203 irradiates the laser beam toward the swivel frame 121 and measures the time until the laser beam is reflected by the swivel frame 121 and returned. The boom undulation angle sensor 203 detects the extension amount of the undulation cylinder 134 based on this time, and detects the undulation angle or the undulation position of the boom 130 with respect to the swivel frame 121 based on the extension amount of the undulation cylinder 134. The boom undulation angle sensor 203 also describes an example, and another sensor that directly detects the undulation angle of the boom 130 by an optical rotary encoder, a potentiometer, or the like may be used.

The boom extension amount sensor 204 is composed of, for example, a laser sensor disposed on the base end boom 131 of the boom 130. The boom extension amount sensor 204 irradiates a laser beam toward the base member 151 of the bucket attachment 150 attached to the tip of the tip boom 132, and measures the time until the laser beam is reflected by the base member 151 and returned. Based on this time, the boom extension amount sensor 204 detects the extension amount of the tip boom 132 with respect to the proximal boom 131 as the extension amount of the boom 130. The boom extension amount sensor 204 also describes an example, and another sensor that directly measures the extension amount of the cable that expands and contracts with the expansion and contraction of the boom may be used.

The bucket swing angle sensor 205 is composed of, for example, a flow rate sensor arranged in an oil passage of the bucket cylinder 153. The bucket swing angle sensor 205 detects the flow rate of the hydraulic oil supplied to the bucket cylinder 153 and calculates the integrated value of the flow rate. The bucket swing angle sensor 205 obtains the extension amount of the piston rod of the bucket cylinder 153 based on this flow rate integrated value, and based on the extension amount of the bucket cylinder 153, the bucket with respect to the base member 151 or the boom 130 of the bucket attachment 150. The swing angle or swing position of 152 is detected. The bucket swing angle sensor 205 is also an example, and the swing angle of the bucket 152 is directly detected by an optical rotary encoder, a potentiometer, or the like, or another sensor that obtains the extension amount of the bucket cylinder 153 by a laser sensor. A sensor may be used.

The outside world sensor 206 is composed of, for example, an RGB-D sensor arranged on the turning frame 121 of the traveling body 110. The outside world sensor 206 acquires RGB or color images of the excavated ground and distance images or point cloud data, and based on these images, includes distance information to the excavated ground, shape information of the excavated ground, or an earth bucket 31 in the work room. The point cloud data 70 of 2 is acquired. As another example of the RGB-D sensor, the external world sensor 206 may use a stereo camera, an ultrasonic rangefinder, a laser sensor, or the like.

Information detected by the traveling body position sensor 201, the turning angle sensor 202, the boom undulation angle sensor 203, the boom extension amount sensor 204, the bucket swing angle sensor 205, and the outside world sensor 206 is transmitted to the main controller 165a of the control unit 165. Will be sent. The main controller 165a includes a traveling body position measuring unit 211, a bucket position measuring unit 212, and a ground shape measuring unit 213.

The traveling body position measuring unit 211 includes distance information from the end portion of the traveling body 4 or the wall portion of the working room 2 to the traveling body 110 detected by the traveling body position sensor 201, and the traveling body position 4 is in the working room 2. The location of the traveling body 110 in the work room 2 is calculated by using the information on the position of the traveling rail provided in the work room 2. Further, the information on the position of the traveling rail 4 in the work room 2 is the information of the traveling rail 4 to which the traveling body 110 is attached, and when the traveling body 110 is attached. It may be set in the traveling body position measuring unit 211. Further, even if the detection of the distance information by the traveling body position sensor 201 is detected for a plurality of surrounding locations, a two-dimensional position in the ceiling of the traveling body 110 or a position including the direction of the traveling body 110 can be detected. good.

The bucket position measuring unit 212 includes a turning angle including the turning direction and position of the turning frame 121 with respect to the traveling frame 111 detected by the turning angle sensor 202, and the undulation of the boom 130 with respect to the turning frame 121 detected by the boom undulating angle sensor 203. Using the angle or undulation position, the extension amount of the boom 130 detected by the boom extension amount sensor 204, and the swing angle or swing position of the bucket 152 with respect to the boom 130 detected by the bucket swing angle sensor 205, The position of the bucket 152 with respect to the traveling frame 111 of the traveling body 110 is calculated.

The ground shape measuring unit 213 determines the position of the traveling body 110 in the work room 2 determined by the traveling body position measuring unit 211, and the turning direction and position of the turning frame 121 with respect to the traveling frame 111 detected by the turning angle sensor 202. Using the included turning angle, the position of the outside world sensor 206 provided on the turning frame 121, the direction in which the distance information is acquired by the outside world sensor 206, and the position of the excavated ground from which the distance information is acquired by the outside world sensor 206 are calculated. do.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 4 is a block diagram showing the overall configuration of the earth bucket recognition system 6 to which the present invention is applied. The earth bucket recognition system 6 recognizes the posture of the earth bucket 31 used in the pneumatic caisson method. The earth bucket recognition system includes the above-mentioned external sensor 206, the remote control device 12 connected to the caisson shovel 100 including the external sensor 206, and the above-mentioned control unit 165 connected to the remote control device 12.

The remote control device 12 is composed of, for example, an electronic device such as a personal computer (PC), but in addition to the PC, it may be any other electronic device such as a mobile phone, a smartphone, a tablet terminal, or a wearable terminal. It may be embodied. The remote control device 12 may recognize the posture of the earth bucket 31 based on the data input from the outside world sensor 206. The remote control device 12 includes an extraction unit 80, a conversion unit 81 connected to the extraction unit 80, and a recognition unit 82 connected to the conversion unit 81.

The extraction unit 80 extracts the point cloud data 72 of the earth bucket 31 from the data input from the external sensor 206. The extraction unit 80 outputs the extracted point cloud data to the conversion unit 81.

The conversion unit 81 converts the point cloud data input from the extraction unit 80 into coordinate data on the three-dimensional space. The conversion unit 81 outputs the coordinate-converted coordinate data to the recognition unit 82.

The recognition unit 82 is based on an error when the coordinate data input from the conversion unit 81 and a mathematical model consisting of a set of coordinates satisfying an equation indicating the side surface of a cylinder in three-dimensional space are superimposed. Recognize 31 postures.

Next, the operation of the earth bucket recognition system 6 to which the present invention is applied will be described with reference to FIG. In step S11, as shown in FIG. 6, the external sensor 206 acquires the point cloud data 70 in the work room 2 including the ground bucket 31. Point cloud data is data consisting of a collection of points having position information in a three-dimensional space. The position information is information that can specify a position, and refers to, for example, coordinates or distance information in a three-dimensional space. The distance information is information indicating the distance to the target.

First, the external sensor 206 uses an RGB-D sensor that acquires point cloud data indicating distance information from the position of the external sensor 206 provided in the work room 2 to the acquisition target, as shown in FIG. 6, for example. The point cloud data 70 in the work room 2 including the earth bucket 31 is acquired. Further, the external sensor 206 is a point cloud data in the work room 2 from which the earth bucket 31 has been removed by the carrier device 32 pulling up the earth bucket 31 to the ground via the material shaft 23 and carrying it out of the work room 2. A certain extraction point cloud data 71 may be acquired. The RGB-D is the same as the position of the RGB-D sensor and the angle and direction of the RGB-D sensor when the point cloud data 71 for extraction and the point cloud data 70 in the work room 2 including the above-mentioned earth bucket 31 are acquired. It is preferable that the data is acquired by using the position of the sensor and the angle and direction of the RGB-D sensor, but this is not the case.

Further, as shown in FIG. 7, the external sensor 206 colors the earth bucket 31 in a color different from that in the work room 2, and acquires the point cloud data 70 in the work room 2 including the above-mentioned earth bucket 31. At the same time, the image 73 may be imaged. At this time, for example, by using an RGB-D sensor, the position of the RGB-D sensor and the angle and direction of the RGB-D sensor when the point cloud data 70 in the work room 2 including the earth bucket 31 are acquired are the same. The image 73 can be simultaneously captured by using the position of the RGB-D sensor and the angle and direction of the RGB-D sensor. The outside world sensor 206 outputs the point cloud data 70, the extraction point cloud data 71, and the captured image 73 in the work room 2 including the acquired earth bucket 31 to the extraction unit 80.

Next, in step S12, the extraction unit 80 extracts the point cloud data 72 of the earth bucket 31 from the point cloud data input from the outside world sensor 206 and the image 73. As a method of extraction by the extraction unit 80, for example, as shown in FIG. 6, the point cloud data 70 in the work room 2 including the earth bucket 31 and the point cloud data 71 for extraction may be used. In this case, for example, as shown in FIG. 6, the point cloud data 72 of the earth bucket 31 is extracted by deleting the above-mentioned extraction point cloud data 71 from the point cloud data 70 in the work room 2 including the above-mentioned earth bucket 31. You may.

Further, as another extraction method by the extraction unit 80, as shown in FIG. 7, the point cloud data 70 in the work room 2 including the earth bucket 31 colored in a color different from that in the work room 2 and the point cloud described above are described. The image 73 captured at the same time as the data 70 may be used. In this case, for example, the same color as the color colored in the earth bucket 31 in the above-mentioned image 73 is extracted, and the point cloud data 70 corresponding to the same position as the position of the extracted image 73 is used as the point cloud data 72 of the earth bucket 31. It may be extracted. The extraction unit 80 outputs the point cloud data 72 of the extracted earth bucket 31 to the conversion unit 81.

Further, in step S12, before extracting the point cloud data of the earth bucket 31, the point cloud data in the range not including the point cloud data of the earth bucket 31 may be removed from the point cloud data 70. To remove the point cloud data, specify the range of the point cloud data of the earth bucket 31 included in the point cloud data 70 from the position information of the outside world sensor 206 and the position information of the earth bucket 31 suspended from the material shaft 23. This may be done by removing the point cloud data outside the range. This makes it possible to shorten the time required for repeatedly extracting the point cloud data of the earth bucket 31.

Further, the timing of starting steps S12 and S11 may be started by inputting a signal from the carrier device 32 in addition to the timing when the user gives a recognition command to the remote control device 12. In this case, the timing of starting recognition may be determined by referring to the value of the load system of the carrier device 32. As a result, it is possible to automatically start recognizing the posture of the earth bucket 31 when recognition is required.

Next, in step S13, the conversion unit 81 performs coordinate conversion of the input point cloud data 72 of the earth bucket 31 into coordinate data on the three-dimensional space. The coordinate data is data indicating the position by three variables such as the distance from each of the x-axis, y-axis, and z-axis in the three-dimensional space. The conversion unit 81 outputs the coordinate-converted coordinate data to the recognition unit 82.

Next, in step S14, the recognition unit 82 recognizes the posture of the earth bucket 31 based on the error when the input coordinate data and the mathematical model are superimposed. A mathematical model is a model consisting of a set of coordinates that satisfy the equations that indicate the sides of a cylinder in three-dimensional space. The posture of the earth bucket 31 indicates how much the earth bucket 31 has translated from the reference position and how much the earth bucket 31 has rotated from the horizontal.

The mathematical model will be described with reference to FIG. As shown in FIG. 8, a cylinder having a bottom surface parallel to the xy plane and having the origin as the center of the bottom surface on the three-dimensional space of the xyz axes that intersect vertically is defined as the reference cylinder 90. When a three-dimensional space centered on the xyz axis was bitch-rotated (y-axis rotation) by an angle θ and then rolled (x-axis rotation) by an angle φ, the x-axis, y-axis, and z-axis changed. Let the things be the x'axis, the y'axis, and the z'axis, respectively. It is not necessary to consider yaw rotation (z-axis rotation) because it does not change even if the cylinder is rotated. It has a bottom surface parallel to the x'y'plane in the three-dimensional space of the x'axis, y'axis, and z'axis that intersect vertically, and further X in the x, y, and z directions from the above-mentioned origin. A cylinder whose center is the point translated by Y and Z is a generalized cylinder 91.

ｒは認識の対象となるアースバケット３１の半径を表す定数であり、ｈは上述したアースバケット３１の軸方向の長さを表す定数である。 A mathematical model is a set of coordinates that satisfy the basic equation of the side surface of the reference cylinder 90, for example, equation (1).

r is a constant representing the radius of the earth bucket 31 to be recognized, and h is a constant representing the axial length of the earth bucket 31 described above.

この一般化円筒９１上の点（ｘ′、ｙ′、ｚ′）を、座標データと仮定し、座標データをｘ方向、ｙ方向、ｚ方向に並進させ、座標軸を回転させ、上述した数学モデルと比較することで、アースバケット３１の姿勢を認識する。 The points (x', y', z') on the generalized cylinder 91 are represented by the equation (2) using the points (x, y, z) on the reference cylinder 90.

The points (x', y', z') on the generalized cylinder 91 are assumed to be coordinate data, the coordinate data is translated in the x, y, and z directions, the coordinate axes are rotated, and the above-mentioned mathematical model is used. By comparing with, the posture of the earth bucket 31 is recognized.

具体的な手順としては、まずステップＳ１７において、Ｘ、Ｙ、Ｚ、θ、φの５つの変数を仮変数として、仮変数を設定する。次にステップＳ１８において、例えば上述したような仮変数と式（３）を用いて、誤差Ｅを算出する。 Using the mathematical model represented by this equation (1) and points (x _p , y _p , z _p ) as coordinate data, the posture of the earth bucket 31 is recognized by the procedure as shown in FIG. For the recognition of the earth bucket 31, for example, using the equation (3) showing the error function, the error E when the point (x _p , y _p , z _p ) of the coordinate data and the mathematical model are superimposed is minimized. This is possible by finding the five variables X, Y, Z, θ, and φ.

As a specific procedure, first, in step S17, a formal variable is set with the five variables X, Y, Z, θ, and φ as formal variables. Next, in step S18, the error E is calculated using, for example, the formal variable as described above and the equation (3).

Next, in step S19, the formal variable is corrected so that the error E becomes smaller based on the error E calculated earlier. After correcting the formal variable in step S19, the process returns to step S17 again, and the error E is calculated in step S18. By repeating the above procedure, a formal variable that minimizes the error E is calculated. The formal variable when the error is the smallest may be the posture of the earth bucket 31. For example, if the error E is calculated as in the equation (4), the minimum error E is the formal variable corrected at the t-th time, so that the formal variable corrected at the t-th time can be used as the attitude of the earth bucket 31. good. For example, if the formal variables when the error E becomes the minimum are X = Y = Z = 2 cm and θ = φ = 5 deg, respectively, the earth bucket 31 moves in the x direction, the y direction, and the z direction from the reference point. It can be recognized that the posture is 5 deg bitch rotation and 5 deg roll rotation from the state of the reference cylinder 90 with the position shifted by 2 cm as the center of the bottom surface.

As a method of calculating the minimum error E, for example, as in equation (5), the calculation is performed by the steepest descent method in which the first formal variable is X = Y = Z = θ = φ = 0 and the objective function is the error E. You may go. Further, in the equation (5), as a method of obtaining Z, for example, Z may be estimated from θ = φ with the lowest point of the coordinate data as 0. Further, for example, when θ = 0, Z = rcosφ, so Z may be estimated.

Further, since the mathematical model and the earth bucket 31 are cylindrical, the yaw rotation can be ignored, so that the calculation time required for repeated calculation can be shortened.

Further, since the earth bucket 31 is suspended from the material shaft 23, bitch rotation and roll rotation hardly occur, so that it is easy to estimate the initial posture, and thus the calculation time required for repeated calculation is shortened. can do.

Further, since the ground bucket 31 is suspended from the material shaft 23, it can be inferred that the ground bucket 31 is directly below the material shaft 23. Therefore, since it is easy to estimate the initial translational postures of X and Y, the calculation time required for repeated calculation can be shortened.

For the reasons mentioned above, it is necessary to recognize the bottom surface or the top surface of the earth bucket only in the Z direction and make an accurate estimation. In this case, by estimating the point having the highest z coordinate in the coordinate data as the upper surface of the earth bucket 31, the calculation time required for the iterative calculation can be shortened.

According to these methods, the above-mentioned calculation method requires repeated calculation of 5 degrees of freedom consisting of 5 variables of X, Y, Z, θ, and φ, so that it is assumed that the calculation takes time. With these methods, it is possible to significantly speed up the calculation.

In the method of recognizing the posture of the earth bucket 31 described above, the mathematical model is the reference cylinder 90 and the coordinate data is the generalized cylinder 91, and the variables of the generalized cylinder 91 are corrected to correct the coordinate data to correct the earth bucket. Recognize 31 postures. Further, the posture of the earth bucket 31 is recognized by correcting the mathematical model by performing the above-mentioned method of recognizing the posture of the earth bucket 31 with the mathematical model as the generalized cylinder 91 and the coordinate data as the reference cylinder 90. May be good.

The recognition unit 82 may output the recognized posture of the earth bucket 31 so that the user can recognize it. As a result, the user can recognize the posture of the earth bucket 31, and when remotely controlling the work of stuffing the earth bucket 31 with earth and sand, the caisson excavator 100 can be operated accurately according to the posture of the earth bucket 31. Is possible.

1 Cason 2 Work room 3 Air supply path 4 Travel rail 6 Earth bucket recognition system 11 Sediment automatic loading device 12 Remote control device 13 Ground remote control room 21 Man shaft 22 Man lock 23 Material shaft 24 Material lock 25 Spiral stairs 31 Earth bucket 32 Carrier device 33 Sediment hopper 41 Air supply pipe 42 Air compressor 43 Air purifier 44 Air supply pressure regulator 45 Automatic decompression device 51 Emergency air compressor 53 Hospital lock 70 Point group data in the work room including the earth bucket 71 For extraction Point group data 72 Point group data of earth bucket 73 Image 80 Extraction unit 81 Conversion unit 82 Recognition unit 90 Reference cylinder 91 Generalized cylinder 100 Cason excavator 110 Traveling body 111 Traveling frame 113 Traveling roller 121 Swivel frame 130 Boom 131 Base end boom 132 Tip boom 133 Telescopic cylinder 134 Undulating cylinder 150 Bucket attachment 151 Base member 152 Bucket 153 Bucket cylinder 165 Control unit 165a Main controller 165b Vehicle controller 165c Boom bucket controller 201 Vehicle position sensor 202 Swing angle sensor 203 Boom undulating angle sensor 204 Boom extension amount sensor 205 Bucket swing angle sensor 206 External world sensor 211 Vehicle position measurement unit 212 Bucket position measurement unit 213 Ground shape measurement unit

Claims (10)

In the earth bucket recognition system that recognizes the posture of the earth bucket in the caisson work room
An acquisition means for acquiring point cloud data indicating distance information from the position of the sensor in the work room to the acquisition target, and
An extraction means for extracting the point cloud data of the earth bucket in the caisson's work room from the point cloud data in the caisson's work room acquired by the acquisition means.
A conversion means for converting the point cloud data of the earth bucket extracted by the extraction means into coordinate data in a three-dimensional space, and a conversion means.
The attitude of the earth bucket based on the error when the mathematical model consisting of a set of coordinates satisfying the equation indicating the side surface of the cylinder in the three-dimensional space and the coordinate data converted by the conversion means are superimposed. An earth bucket recognition system characterized by having a recognition means for recognizing. The recognition means corrects and corrects the mathematical model by rotating it around a coordinate axis in three-dimensional space so that the error when the mathematical model and the coordinate data are superimposed becomes smaller. The earth bucket recognition system according to claim 1, wherein the calculation of an error when the mathematical model and the coordinate data are superimposed is repeated. The recognition means corrects the coordinate data by rotating it around a coordinate axis in a three-dimensional space so that the error when the mathematical model and the coordinate data are superimposed becomes smaller, and corrects the mathematics. The earth bucket recognition system according to claim 1 or 2, wherein the calculation of an error when the model and the corrected coordinate data are superimposed is repeated. The recognition means is rotated around a coordinate axis in three-dimensional space in order to correct the mathematical model or the coordinate data when the error when the mathematical model and the coordinate data are superimposed is minimized. The earth bucket recognition system according to claim 2 or 3, wherein the attitude of the earth bucket is recognized based on an angle. The recognition means translates the mathematical model in a three-dimensional space, corrects the mathematical model, and corrects the mathematical model so that the error when the mathematical model and the coordinate data are superimposed becomes smaller. The earth bucket recognition system according to any one of claims 1 to 4, wherein the calculation of an error when superimposing the coordinate data is repeated. The recognition means translates the coordinate data in a three-dimensional space and corrects the coordinate data so that the error when the mathematical model and the coordinate data are superimposed becomes smaller, and corrects the mathematical model. The earth bucket recognition system according to any one of claims 1 to 5, wherein the calculation of an error when superimposing the coordinate data is repeated. The recognition means is based on the position translated in three-dimensional space to correct the mathematical model or the coordinate data when the error when superimposing the mathematical model and the coordinate data is minimized. The earth bucket recognition system according to claim 5 or 6, wherein the earth bucket recognizes the posture of the earth bucket. The acquisition means acquires the extraction point cloud data in the work room from which the earth bucket has been removed in advance.
The extraction means is characterized in that the point cloud data of the earth bucket is extracted by deleting the point cloud data for extraction acquired by the acquisition means from the point cloud data in the work room where the earth bucket is located. The earth bucket recognition system according to any one of claims 1 to 7. The acquisition means images the working room of the caisson and obtains images.
The extraction means refers to the color of the earth bucket from the point cloud data acquired by the acquisition means, and extracts the point cloud data according to the position of the color in the image captured by the acquisition means. The earth bucket recognition system according to any one of claims 1 to 8, wherein the point cloud data of the earth bucket is extracted. In the earth bucket recognition program that recognizes the posture of the earth bucket in the caisson work room
The acquisition step of acquiring point cloud data indicating the distance information from the position of the sensor in the work room to the acquisition target, and
An extraction step of extracting the point cloud data of the earth bucket in the caisson's work room from the point cloud data of the caisson's work room acquired by the acquisition step.
A conversion step for converting the point cloud data of the earth bucket extracted in the extraction step into coordinate data in three-dimensional space, and a conversion step.
The attitude of the earth bucket based on the error when the mathematical model consisting of a set of coordinates satisfying the equation indicating the side surface of the cylinder in the three-dimensional space and the coordinate data coordinated by the conversion step are superimposed. An earth bucket recognition program characterized by having a computer perform a recognition step that recognizes.

Images