JP7386592B2 - Construction machinery operation assistance system - Google Patents

Description

The present invention relates to an operation assistance system that assists an operator in operating a construction machine.

Conventionally, technology has been developed that uses a three-dimensional laser scanner to measure and display the shape of the ground, etc. that is the target of work, for the purpose of assisting operators in operating construction machinery and assisting unmanned remote control of construction machinery. ing.
For example, Patent Document 1 listed below discloses an imaging unit that is mounted on a construction machine and acquires an image of a work target area, and an imaging unit that is mounted on a construction machine and projects a light beam onto a target object to obtain a set of point cloud data. A three-dimensional scanner that acquires dimensional distance data, an operation input unit that remotely controls construction machinery by wireless, a captured image display unit installed near the operation input unit that displays images acquired by the imaging unit, and an operation input unit that displays images acquired by the imaging unit. A remote control device is disclosed that includes a scanner image display section that is installed near an input section and displays three-dimensional distance data acquired by a three-dimensional scanner.

Japanese Patent Application Publication No. 2015-043488

However, in addition to being expensive, 3D laser scanners have the problem that most of them do not have the durability to withstand use at civil engineering sites where vibrations, dust, wind and rain are a major factor. .
The present invention has been made in view of these circumstances, and its purpose is to provide an operation assistance system for construction machinery that can grasp the three-dimensional shape of a work object at low cost.

In order to achieve the above-mentioned object, the present invention includes: a rotating section for rotating a construction machine including a work member for deforming a measurement target in a first direction; a two-dimensional laser scanner that scans a laser beam in a second direction perpendicular to the direction and measures a distance to a portion of the object to be measured along the second direction; and a rotating section that rotates the construction machine. a surface shape calculation unit that calculates three-dimensional data of the surface shape of the object to be measured by continuously obtaining detection values of the two-dimensional laser scanner; and a surface shape calculation unit that calculates three-dimensional data of the surface shape of the measurement target object; and a display control unit that displays the three-dimensional data of and the current position of the work member on a monitor provided at a position visible to an operator of the construction machine , the construction machine The display controller further includes a target shape acquisition unit that is operated to transform the object into a predetermined target shape and acquires three-dimensional data of the target shape, and the display control unit is configured to acquire the three-dimensional data of the target shape and the three-dimensional data of the target shape. Based on three-dimensional data of the surface shape of the measurement target before deformation and three-dimensional data of the surface shape of the measurement target after deformation, a three-dimensional image of the target shape and the three-dimensional data of the surface shape of the measurement target before deformation are generated. A three-dimensional image of the surface shape and a three-dimensional image of the surface shape after the deformation are displayed side by side on the monitor without being superimposed.
The present invention further includes a difference calculation unit that calculates a difference amount between the three-dimensional data of the target shape and the three-dimensional data of the surface shape after the deformation , and the display control unit calculates the difference amount as a heat map. It is characterized by displaying.
The present invention further comprises: the three-dimensional data of the target shape includes coordinate data of each point in reference coordinates, and further comprises a machine position information calculation unit that calculates position information of the construction machine in the reference coordinates; The surface shape calculation section calculates coordinate data in the reference coordinates for each point of the three-dimensional data of the surface shape based on the position information of the construction machine, and calculates coordinate data in the reference coordinates for each point of the three-dimensional data of the surface shape, and Displaying an image, a three-dimensional image of the surface shape before the deformation, and a three-dimensional image of the surface shape after the deformation on the monitor is performed using the coordinate data of the three-dimensional data of the surface shape and the target. It is characterized in that it is performed using coordinate data of three-dimensional data of the shape.
The present invention is characterized in that the machine position information calculation unit calculates the position information of the construction machine at the reference coordinates based on signals received by two satellite positioning system receivers attached to the construction machine. shall be.
In the present invention, the machine position information calculation unit calculates the position of the construction machine based on a signal received by a single satellite positioning system receiver attached to the construction machine and azimuth information centered on the construction machine. The method is characterized in that position information at the reference coordinates is calculated.

According to the present invention, three-dimensional data of the surface shape of the object to be measured is obtained by rotating a construction machine equipped with a two-dimensional laser scanner, so the object to be measured can be measured at a lower cost than when using a three-dimensional laser scanner. Be able to grasp the three-dimensional shape of objects. Furthermore, since two-dimensional laser scanners have higher durability than three-dimensional laser scanners, it is possible to construct an operation assistance system for construction machinery that can withstand use at civil engineering sites where vibrations and other effects are significant.
Further, according to the present invention, the three-dimensional data of the surface shape of the object to be measured and the current position of the work member are displayed on the monitor, so the operator can use only the monitor without directly viewing the object to be measured. This makes it possible to perform the work, which is advantageous in improving work efficiency.
Further, according to the present invention , the worker can intuitively grasp the location and amount of work that requires deformation work, which is advantageous in improving work efficiency.
Furthermore, according to the present invention, the amount of difference between the three-dimensional data of the target shape and the three-dimensional data of the current measurement target is displayed in a color-coded heat map, so that the worker can easily understand the amount of work required to reach the target shape. It can be grasped intuitively, which is advantageous in improving work efficiency.
Further, according to the present invention, since the three-dimensional data is superimposed using the coordinate data of the three-dimensional data of the surface shape of the measurement object and the coordinate data of the three-dimensional data of the target shape, the three-dimensional data is more accurately can be aligned.
Further, according to the present invention, the position information of the construction machine is calculated based on the signals received by the two satellite positioning system receivers, so the position of the construction machine can be accurately calculated by following the change in position due to turning. It will be advantageous above.
Further, according to the present invention, the position information of the construction machine is calculated based on the signal and azimuth information received by the satellite positioning system receiver, so the position of the construction machine can be accurately calculated by following the change in position due to turning. It is advantageous to do so.

It is a side view of the backhoe 10 in which the operation assistance system 30 is mounted. FIG. 1 is a plan view of a backhoe 10. FIG. 3 is an explanatory diagram showing a scanning direction of a two-dimensional laser scanner 302. FIG. FIG. 3 is an explanatory diagram showing a turning direction of a two-dimensional laser scanner 302. FIG. FIG. 3 is a diagram illustrating an example of superimposed display of three-dimensional data. 3 is a block diagram showing the configuration of an operation assistance system 30. FIG.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a construction machine operation assistance system (hereinafter referred to as "operation assistance system") according to the present invention will be described in detail below with reference to the accompanying drawings.
First, a construction machine equipped with an operation assistance system according to the present invention will be explained. In this embodiment, a case where the construction machine is a backhoe operated by an operator will be described.

First, the configuration of the backhoe will be explained.
As shown in FIGS. 1 and 2, the backhoe 10 includes a lower traveling body 12, an upper revolving body 14, a boom 16, an arm 18, and a bucket 20.

The lower traveling body 12 travels on the ground G due to the rotation of the crawler 1202.
The upper revolving body 14 is provided on the upper part of the lower traveling body 12 so as to be horizontally swingable about a pivot axis. That is, the upper rotating body 14 can rotate in the horizontal direction H with respect to the lower traveling body 12 and the ground G by the rotating mechanism (swivel portion) 13 (see FIG. 2).
The upper revolving body 14 is provided with an operation room 1402, and the operation room 1402 is configured to control the running of the lower revolving body 12, the rotation of the upper revolving body 14, the swinging of the boom 16, the swinging of the arm 18, and the swinging of the bucket 20. A plurality of operating devices such as operating levers and operating pedals (all not shown) are installed for operating the devices.
Further, a monitor 28 (see FIG. 5) is installed in the operation room 1402, and information to assist the operator in operating the backhoe 10 is displayed.
Further, on the top surface of the operation room 1402, a two-dimensional laser scanner 302 and a GNSS receiver 304, which will be described later, are placed. Note that the two-dimensional laser scanner 302 and the GNSS receiver 304 are not limited to the top surface of the operation room 1402, but may be placed at any position that does not interfere with measurement and reception in each device.

The base end of the boom 16 is swingably supported by the upper revolving body 14 via a support shaft extending in the horizontal direction.
The arm 18 has its base end swingably supported at the tip of the boom 16 via a support shaft extending in the horizontal direction.
The bucket 20 has its base end swingably supported at the tip of the arm 18 via a support shaft extending in the horizontal direction. The bucket 20 corresponds to a work member that performs excavation, that is, deformation work of the soil, which is the object to be measured in this embodiment.
A boom cylinder 1602 for swinging the boom 16 is provided between the upper revolving body 14 and the boom 16.
An arm cylinder 1802 for swinging the arm 18 is provided between the boom 16 and the arm 18.
A bucket cylinder 2002 for swinging the bucket 20 is provided between the arm 18 and the bucket 20.
These boom cylinder 1602, arm cylinder 1802, and bucket cylinder 2002 are hydraulic cylinders.
Therefore, as the boom cylinder 1602 expands and contracts, the boom 16 swings relative to the upper revolving structure 14.
Furthermore, the arm 18 is swung relative to the boom 16 by expanding and contracting the arm cylinder 1802.
Furthermore, the bucket 20 is swung relative to the arm 18 by expanding and contracting the bucket cylinder 2002 .

In this embodiment, an operator in the operation room 1402 operates each part of the backhoe 10 to perform excavation work (deformation work) on the soil to be worked located on the side where the front of the backhoe 10 (bucket 20) is located. conduct. In the operation assistance system 30, which will be described later, the surface shape of the soil to be worked is calculated as the measurement target T.

Next, the operation assistance system 30 that assists the operation of the backhoe 10 will be explained.
FIG. 5 is a block diagram showing the configuration of the operation assistance system 30.
The operation assistance system 30 is mainly composed of sensors and a computer 40 attached to the backhoe 10, which is a construction machine.
A two-dimensional laser scanner 302, a GNSS (Global Navigation Satellite System) receiver 304, and a tilt sensor 306 are attached to the backhoe 10.

The two-dimensional laser scanner 302 measures the distance to the laser irradiation position by emitting laser light L in a predetermined direction and receiving reflected light. In this embodiment, the two-dimensional laser scanner 302 scans the laser beam L in the direction V perpendicular to the ground G, as shown in FIG. ) to each point. That is, when the backhoe 10 is stationary, the distance of one line along the vertical direction V of the object T to be measured can be measured.
Here, the upper revolving body 14 of the backhoe 10 is provided so as to be rotatable in the horizontal direction H with respect to the ground G. Therefore, as shown in FIG. 4, by scanning the two-dimensional laser scanner 302 in the vertical direction V while rotating the upper rotating body 14 in the horizontal direction H, the distance to each point on the entire measurement target T is measured. be able to.
That is, the two-dimensional laser scanner 302 is attached to the backhoe 10, which is a construction machine, and emits a laser beam in a vertical direction V (second direction) orthogonal to a horizontal direction H (first direction), which is the turning direction of the backhoe 10. L is scanned and the distance to the part of the measurement target T along the vertical direction V is detected.

A GNSS receiver (satellite positioning system receiver) 304 is installed on the top of the backhoe 10 and receives signals from GNSS satellites. A machine position information calculation unit 402, which will be described later, calculates the position coordinates of the GNSS receiver 304 of the backhoe 10 based on the signal received by the GNSS receiver 304.
In this embodiment, two GNSS receivers 304 are installed on the top surface of the backhoe 10, as shown in FIG. This is to accurately detect changes in position due to turning of the backhoe 10. By installing two GNSS receivers 304 and calculating position coordinates at two locations, the turning state (yaw angle) of the backhoe 10 can be detected with high accuracy.
Note that instead of installing two GNSS receivers 304, one GNSS receiver 304 and one compass may be installed as described later, or a GNSS receiver 304 with a built-in compass may be installed. .

The tilt sensor 306 detects the tilt of the backhoe 10.
In this embodiment, it is assumed that a tilt sensor 306 that detects the roll angle and pitch angle of the backhoe 10 is attached.

The computer 40 includes a CPU, a ROM that stores and stores control programs, a RAM that serves as an operating area for the control programs, an EEPROM that holds various types of data in a rewritable manner, an interface unit that interfaces with peripheral circuits, etc. be done.
The computer 40 functions as a machine position information calculation section 402, a surface shape calculation section 404, a target shape acquisition section 406, a difference calculation section 408, and a display control section 410 when the CPU executes the control program.

In addition, in the following description, the reference coordinates are coordinates arbitrarily set in the space where the backhoe 10 is located. On the other hand, absolute coordinates are coordinates used when calculating position information based on a signal received by the GNSS receiver 304, and are expressed by latitude, longitude, and height, for example. The amount of deviation between the reference coordinates and the absolute coordinates is known, and positional information on one coordinate can be mutually converted into positional information on the other coordinate.
Further, the positional information of the backhoe 10 is, for example, the positional information of the laser light receiving part (reference position) O of the two-dimensional laser scanner 302.

The machine position information calculation unit 402 calculates position information of the backhoe 10 (construction machine) in reference coordinates.
In the present embodiment, machine position information calculation section 402 calculates position information of backhoe 10 in reference coordinates based on signals received by two GNSS receivers 304 attached to backhoe 10. First, based on the signals received by the two GNSS receivers 304, position information in absolute coordinates of each GNSS receiver 304 is calculated, and further converted into position information on reference coordinates.
Next, the position information of the laser light receiving unit O in the reference coordinates is calculated from the position information of the two GNSS receivers 304. Since the mounting position of the GNSS receiver 304 is known, by setting the amount of positional deviation between the laser receiver O and the GNSS receiver 304 as an offset value, the laser receiver O can be determined from the position information of the GNSS receiver 304. location information can be calculated.
Furthermore, the machine position information calculation unit 402 detects the inclination of the backhoe 10 from the detection result of the inclination sensor 306 and corrects each position information.

In this embodiment, two GNSS receivers 304 are installed, and the turning state (yaw angle) of the backhoe 10 can be calculated with high accuracy from the relative change amount of the position information of these two GNSS receivers 304. .
Note that instead of installing two GNSS receivers 304, the turning state of the backhoe 10 can be calculated by installing a single GNSS receiver 304 and a direction meter that measures the direction around the backhoe 10. You may. In this case, the GNSS receiver 304 may have a built-in compass, or the GNSS receiver 304 and the compass may be installed independently.

The surface shape calculation unit 404 calculates three-dimensional data of the surface shape of the measurement target T by rotating the backhoe 10 using the rotation mechanism 13 and continuously obtaining detection values of the two-dimensional laser scanner 302. As described above, the two-dimensional laser scanner 302 scans the laser beam L in the vertical direction V perpendicular to the turning direction (horizontal direction H) of the backhoe 10, and scans the part of the measurement target T along the vertical direction V. The distance (in this embodiment, the distance from the laser light receiving section O) is detected. The surface shape calculation unit 404 can continuously detect the distance to each point of the measurement target T also in the horizontal direction H as the backhoe 10 turns. Then, each point on the measurement object T is determined based on the distance to each point on the measurement object T and the position information (coordinate data in the reference coordinates) of the laser light receiving section O calculated by the machine position information calculation section 402. Calculate coordinate data (current actual coordinate data) at the reference coordinates.
Furthermore, during work using the backhoe 10, the surface shape calculation unit 404 sequentially scans with the two-dimensional laser scanner 302 and updates the three-dimensional data of the surface shape of the measurement target T.
The three-dimensional data calculated by the surface shape calculation unit 404 is recorded on a hard disk (not shown) or the like along with information on the time at which the scan was performed. This allows, for example, to calculate three-dimensional data of the surface shape of the measurement target T before work and compare it with the surface shape after work for a certain period of time (for example, one work unit day), or to calculate the amount of work within a certain period of time. (differences in three-dimensional data).

The target shape acquisition unit 406 acquires three-dimensional data of the target shape of the measurement target object T.
In this embodiment, a backhoe 10 is used to transform the measurement target T (work target soil) into a predetermined target shape. The three-dimensional data of the target shape corresponds to the design data of this work. The three-dimensional data (design data) of the target shape includes coordinate data (target coordinate data) of each point of the measurement target T in reference coordinates.
The target shape acquisition unit 406 reads three-dimensional data of the target shape from, for example, a hard disk device (not shown), or receives three-dimensional data of the target shape via a network.

The difference calculation unit 408 calculates the amount of difference between the three-dimensional data of the target shape and the three-dimensional data of the surface shape of the measurement target T. That is, the difference calculation unit 408 calculates the difference between the shape of the soil in the design data and the shape of the soil at the current time. The difference is calculated for each point of the three-dimensional data, for example, in the form of 〇cm△mm. This difference becomes the amount of work (amount of excavation) to be performed by the backhoe 10.

The display control unit 410 displays the three-dimensional data of the surface shape of the measurement target T calculated by the surface shape calculation unit 404 on the monitor 28 provided at a position where the operator of the backhoe 10 can view the data.
More specifically, the display control unit 410 displays three-dimensional data (design data) of the target shape and three-dimensional data of the surface shape of the measurement target T in a superimposed manner. As described above, since the three-dimensional data (design data) of the target shape includes the coordinate data in the reference coordinates, the display control unit 410 uses the coordinate data of the three-dimensional data of the surface shape of the measurement target T and , the coordinate data of the three-dimensional data of the target shape are used to superimpose the three-dimensional data of the surface shape and the three-dimensional data of the target shape. By performing such a superimposed display, the operator can intuitively recognize which area of the measurement target T should be excavated, and the efficiency of the work can be improved.

FIG. 6 is a diagram showing an example of superimposed display of three-dimensional data. FIG. 6A is a state in which three three-dimensional data D1 to D3 are superimposed and displayed, and FIG. 6B is a state in which three three-dimensional data D1 to D3 are displayed in a superimposed manner. They are displayed individually.
The three-dimensional data D1 indicates the shape of the measurement target T before work, the three-dimensional data D2 indicates the current shape of the measurement target T, and the three-dimensional data D3 indicates the shape of the measurement target T in the design data.
By superimposing and displaying these, it is possible to intuitively grasp the amount of work to date, the amount of work to achieve the target shape, the areas to be worked on, etc.
Note that instead of superimposing all three three-dimensional data D1 to D3, the combination can be freely selected, for example, by superimposing three-dimensional data D2 indicating the current shape and three-dimensional data D3 indicating the shape of the design data. .
Furthermore, since the shape of the measurement target T is calculated using three-dimensional data, the viewpoint of the displayed image can also be switched arbitrarily.

Further, at this time, the amount of difference between the three-dimensional data of the target shape and the three-dimensional data of the surface shape calculated by the difference calculation unit 408 may be displayed as a heat map. That is, for example, if the difference amount is less than 2 cm and there is no need to excavate, it will be displayed in red, if the difference amount is 2 cm or more and less than 4 cm, it will be displayed in orange, and if the difference amount is 4 cm or more and less than 6 cm, it will be displayed in yellow. This allows the worker to intuitively recognize which area and how much to excavate, further improving work efficiency.

Furthermore, the display control unit 410 displays on the monitor 28 the current position of the bucket 20, which is a work member that performs excavation (deformation) work in the backhoe 10, along with three-dimensional data of the surface shape of the measurement target T. Good too. Thereby, the operator only has to operate the backhoe 10 while visually viewing the monitor 28, and the work efficiency can be further improved.
Note that the current position of the bucket 20 can be calculated from the current position and posture of the backhoe 10, the operating amounts of various cylinders, and the like.

As described above, the operation assistance system 30 according to the embodiment obtains three-dimensional data of the surface shape of the measurement target T by rotating the backhoe 10 equipped with the two-dimensional laser scanner 302, so the three-dimensional laser The three-dimensional shape of the object to be measured can be grasped at a lower cost than when using a scanner. Furthermore, since the two-dimensional laser scanner 302 has higher durability than the three-dimensional laser scanner, it is possible to construct an operation assistance system for construction machinery that can withstand use at civil engineering sites where vibrations and the like are significant.
In addition, the operation assistance system 30 superimposes and displays the three-dimensional data of the target shape (design data) and the three-dimensional data of the current measurement target T, so that the operator can easily see where the deformation work is required and the amount of work. can be grasped intuitively, which is advantageous in improving work efficiency. At this time, if the amount of difference between the 3D data of the target shape and the 3D data of the current measurement target T is displayed in a color-coded heat map, the worker can more intuitively understand the amount of work required to reach the target shape. This is advantageous in improving work efficiency.
In addition, if the operation assistance system 30 displays the current position of the bucket 20, which is a work member, on the monitor 28 together with the three-dimensional data of the surface shape of the object to be measured T, the operator can directly visually observe the object to be measured. This makes it possible to perform work using only the monitor without having to use the monitor, which is advantageous in improving work efficiency.
In addition, since the operation assistance system 30 superimposes the three-dimensional data using the coordinate data of the three-dimensional data of the surface shape of the measurement target T and the coordinate data of the three-dimensional data of the target shape, Data can be aligned.
In addition, since the operation assistance system 30 calculates the position information of the backhoe 10 based on the signals received by the two GNSS receivers 304, it is possible to accurately calculate the position of the backhoe 10 by following changes in position due to turning. It is advantageous.

In addition, in this embodiment, the backhoe 10 is provided with the swing mechanism 13 as a construction machine and the upper swing structure 14 can swing relative to the lower traveling structure 12. However, the construction machine is not limited to this, for example. The scanning position in the horizontal direction may be changed by placing the scanner on a swivel table and rotating the swivel table using the swivel table. By doing so, the present invention can be applied to construction machines that are not provided with the swing mechanism 13.

Further, in this embodiment, the case where the operator operates the backhoe 10 while riding on the backhoe 10 has been described, but the invention is not limited to this. The present invention is also applicable to the case where the backhoe 10 is operated by control.

10 Backhoe 12 Lower traveling body 13 Swivel mechanism 14 Upper rotating body 16 Boom 18 Arm 20 Bucket 28 Monitor 30 Operation assistance system 302 3D laser scanner 304 GNSS receiver 306 Inclination sensor 40 Computer 402 Machine position information calculation section 404 Surface shape calculation section 406 Target shape acquisition section 408 Difference calculation section 410 Display control section G Ground L Laser light O Laser light receiving section T Measurement object H Horizontal direction V Vertical direction

Claims (5)

a turning unit that turns a construction machine equipped with a work member that performs a work of deforming a measurement target in a first direction;
a two-dimensional laser scanner that is attached to the construction machine and scans a laser beam in a second direction perpendicular to the first direction to measure a distance to a part of the measurement target along the second direction; and,
a surface shape calculating section that calculates three-dimensional data of the surface shape of the measurement target by rotating the construction machine using the rotating section and continuously obtaining detection values of the two-dimensional laser scanner;
a display control unit that displays the three-dimensional data of the surface shape calculated by the surface shape calculation unit and the current position of the work member on a monitor installed at a position visible to an operator of the construction machine; , comprising:
The construction machine is operated to transform the measurement object into a predetermined target shape,
further comprising a target shape acquisition unit that acquires three-dimensional data of the target shape,
The display control unit is configured to display information based on three-dimensional data of the target shape, three-dimensional data of the surface shape of the measurement target before deformation, and three-dimensional data of the surface shape of the measurement target after deformation. displaying a three-dimensional image of the target shape, a three-dimensional image of the surface shape before the deformation, and a three-dimensional image of the surface shape after the deformation side by side without superimposing them on the monitor;
An operation assistance system for construction machinery, which is characterized by: further comprising a difference calculation unit that calculates a difference amount between the three-dimensional data of the target shape and the three-dimensional data of the surface shape after the deformation ,
The display control unit displays the difference amount as a heat map.
2. The construction machine operation assistance system according to claim 1 . The three-dimensional data of the target shape includes coordinate data of each point in reference coordinates,
further comprising a machine position information calculation unit that calculates position information of the construction machine in the reference coordinates,
The surface shape calculation unit calculates coordinate data in the reference coordinates for each point of the three-dimensional data of the surface shape based on the position information of the construction machine,
The display on the monitor of the three-dimensional image of the target shape, the three-dimensional image of the surface shape before the deformation, and the three-dimensional image of the surface shape after the deformation by the display control unit is based on the surface shape. The coordinate data of the three-dimensional data of the target shape and the coordinate data of the three-dimensional data of the target shape are used.
The operation assistance system for construction machinery according to claim 1 or 2, characterized in that: The machine position information calculation unit calculates position information of the construction machine at the reference coordinates based on signals received by two satellite positioning system receivers attached to the construction machine.
4. The operation assistance system for construction machinery according to claim 3 . The machine position information calculation unit is configured to calculate the location of the construction machine at the reference coordinates based on a signal received by a single satellite positioning system receiver attached to the construction machine and azimuth information centered on the construction machine. Calculate location information,
4. The operation assistance system for construction machinery according to claim 3 .

Images