JP2021188258A - System for shovel - Google Patents

Abstract

To provide a system for a shovel capable of further appropriately supporting loading work.SOLUTION: A construction support system SYS is the system for a shovel 100 having a lower traveling body 1, an upper turning body 3 turnably loaded on the lower traveling body 1 and excavation attachments attached to the upper turning body 3. The system SYS is configured to acquire three-dimensional data related to a dump truck 200 on the basis of the output of a space recognition device S6 as an object detection device, and to estimate at least one of the position, direction and size of the dump truck 200, on the basis of information related to at least one face composing a platform of the dump truck 200.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a system for excavators.

Conventionally, a shovel that recognizes the position of the rear corner of the loading platform of a dump truck in order to prevent the excavator attachment of the excavator from coming into contact with the dump truck is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-21290

However, the excavator described above only locally recognizes the position of the rear corner of the dump truck carrier, and does not recognize the size of the dump truck carrier. Therefore, it is not possible to properly support the loading work.

Therefore, it is desirable to provide a system for excavators that can more appropriately support the loading work.

The system for an excavator according to an embodiment of the present invention is for an excavator having a lower traveling body, an upper swivel body rotatably mounted on the lower traveling body, and an attachment attached to the upper swivel body. The system obtains three-dimensional data about the dump truck based on the output of the object detection device, and the position, orientation, and orientation of the dump truck are based on the information about at least one surface constituting the loading platform of the dump truck. Estimate at least one of the sizes.

The excavator system described above can better support the loading operation.

It is a figure which shows the configuration example of a construction support system. It is a top view of the excavator. It is a figure which shows the configuration example of the drive system mounted on an excavator. It is a flowchart which shows an example of the estimation process. It is a figure which shows the example of the data used in the estimation process. It is a top view of the excavator and the dump truck at the construction site. It is a flowchart which shows another example of the estimation process.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In each drawing, the same or corresponding configurations are designated by the same or corresponding reference numerals and the description thereof will be omitted.

FIG. 1 is a diagram showing a configuration example of a construction support system SYS, which is an example of a system for a shovel according to an embodiment of the present invention. The construction support system SYS is configured to recognize the position, size, orientation, etc. of the dump truck 200 as a transport vehicle, and to support the loading work by the excavator (excavator 100) as a construction machine. .. FIG. 2 is a top view of the excavator 100. FIG. 3 is a diagram showing a configuration example of a drive system mounted on the excavator 100. The drive system is a system for driving the excavator 100.

First, the outline of the excavator 100 will be described with reference to FIGS. 1 and 2. The excavator 100 includes a lower traveling body 1, a swivel mechanism 2, an upper swivel body 3, a boom 4, an arm 5, a bucket 6, a cabin 10, and the like. The upper swivel body 3 is mounted on the lower traveling body 1 so as to be swivelable via the swivel mechanism 2. The boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of the attachment. The attachment may be a lifting magnet attachment, a grapple attachment, or the like.

As shown in FIG. 2, the lower traveling body 1 has a crawler 1C including a left crawler 1CL and a right crawler 1CR. The crawler 1C is driven by the traveling hydraulic motor 2M. The traveling hydraulic motor 2M includes a left traveling hydraulic motor 2ML for driving the left crawler 1CL and a right traveling hydraulic motor 2MR for driving the right crawler 1CR.

The upper swing body 3 is driven by the swing hydraulic motor 2A. The swing hydraulic motor 2A can change the direction of the upper swing body 3 with respect to the lower traveling body 1. The upper swivel body 3 may be electrically driven by a swivel motor. The swing hydraulic motor 2A and the swing motor may be used in combination.

The boom 4 is rotatably attached to the center of the front portion of the upper swing body 3. An arm 5 is rotatably attached to the tip of the boom 4, and a bucket 6 as an end attachment is rotatably attached to the tip of the arm 5. The boom 4 is driven by the boom cylinder 7. The arm 5 is driven by the arm cylinder 8. The bucket 6 is driven by the bucket cylinder 9.

Depending on the work content and the like, another end attachment may be attached to the tip of the arm 5 instead of the bucket 6 which is an example of the end attachment. Other end attachments are, for example, lifting magnets or grapples.

The cabin 10 is a driver's cab on which the operator boarded. The cabin 10 is mounted on the front left side of the upper swing body 3. If the excavator 100 is an unmanned excavator, the cabin 10 may be omitted.

Next, the function of the excavator 100 will be described. As shown in FIG. 3, the drive system of the excavator 100 includes an engine 11, a regulator 13, a main pump 14, a control valve unit 17, and the like. As shown in FIG. 2, the hydraulic drive system of the excavator 100 includes hydraulic actuators such as a traveling hydraulic motor 2M, a turning hydraulic motor 2A, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

The engine 11 is a power source for the hydraulic drive system, and is mounted on the rear portion of the upper swing body 3, for example. Specifically, the engine 11 rotates at a preset target rotation speed under direct or indirect control by the shovel controller 30, which will be described later, to drive the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses light oil as fuel.

A dial 75 is provided in the cabin 10 of the excavator 100. The dial 75 is a dial for adjusting the engine speed, and is used, for example, for stepwise switching the engine speed. In the present embodiment, the dial 75 is provided so that the engine speed can be switched in four stages of SP mode, H mode, A mode, and IDLE mode. The dial 75 is configured to send data indicating the setting state of the engine speed to the excavator controller 30. Note that FIG. 3 shows a state in which the H mode is selected by the dial 75.

The SP mode is a rotation speed mode selected when it is desired to prioritize the amount of work, and the highest engine rotation speed is used. The H mode is a rotation speed mode selected when it is desired to achieve both a work amount and a fuel consumption, and the second highest engine rotation speed is used. The A mode is a rotation speed mode selected when it is desired to operate the excavator with low noise while giving priority to fuel consumption, and uses the third highest engine rotation speed. The IDLE mode is a rotation speed mode selected when the engine is desired to be in an idling state, and the lowest engine rotation speed is used. The engine 11 is controlled to a constant rotation speed at the engine rotation speed corresponding to the rotation speed mode set by the dial 75.

The main pump 14 is configured to discharge hydraulic oil. Like the engine 11, the main pump 14 is mounted on the rear part of the upper swing body 3 and supplies hydraulic oil to the control valve unit 17 through the hydraulic oil line. The main pump 14 is driven by the engine 11. In the present embodiment, the main pump 14 is a swash plate type variable displacement hydraulic pump, and the discharge flow rate (push-out volume per rotation) is controlled under the control of the shovel controller 30.

The regulator 13 is configured to control the discharge amount of the main pump 14. In the present embodiment, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the excavator controller 30.

The oil temperature sensor 14c provided in the pipeline between the tank in which the hydraulic oil sucked by the main pump 14 is stored and the main pump 14 transmits data indicating the temperature of the hydraulic oil flowing in the pipeline to the excavator controller 30. It is configured to send.

The control valve unit 17 is a hydraulic control device including a plurality of control valves. The control valve unit 17 is mounted on the central portion of the upper swing body 3, for example, and is configured to control the hydraulic drive system in response to the operation of the operator with respect to the operating device 26. The control valve unit 17 is connected to the main pump 14 via a hydraulic oil line, and the hydraulic oil supplied from the main pump 14 is supplied to a plurality of hydraulic actuators (running hydraulic motor 2M, depending on the operating state of the operating device 26). It selectively supplies one or a plurality of the swing hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9). Specifically, the control valve unit 17 includes a plurality of control valves (spool valves) that control the flow rate and flow direction of the hydraulic oil supplied from the main pump 14 to each of the plurality of hydraulic actuators.

The operating system of the excavator 100 includes a pilot pump 15 and an operating device 26. Further, the operation system of the excavator 100 includes a shuttle valve 32 as a configuration related to a machine control function (described later) by the excavator controller 30.

The pilot pump 15 is mounted on the rear portion of the upper swing body 3, for example, and is configured to be able to supply hydraulic oil to equipment such as an operating device 26 via a pilot line. In the present embodiment, the pilot pump 15 is a fixed-capacity hydraulic pump and is driven by the engine 11.

The operating device 26 is provided near the cockpit in the cabin 10. The operation device 26 is an operation input means used for operating the operated portion (lower traveling body 1, upper turning body 3, boom 4, arm 5, bucket 6, etc.). That is, the operating device 26 is used for operating the hydraulic actuators (left traveling hydraulic motor 2ML, right traveling hydraulic motor 2MR, swivel hydraulic motor 2A, boom cylinder 7, arm cylinder 8, bucket cylinder 9, etc.) for driving the operated portion. It is an operation input means used. The operating device 26 includes an operating lever and a traveling lever. Each of the operating devices 26 is connected to a corresponding control valve in the control valve unit 17 via a shuttle valve 32 provided on the secondary pilot line. As a result, the pilot pressure corresponding to the operating state of the corresponding operating device 26 is input to each control valve. Therefore, each control valve operates according to the operation state of the corresponding operation device 26, and the hydraulic actuator is driven according to the operation of the corresponding control valve. The operating lever includes a left operating lever and a right operating lever. The left operating lever is used, for example, to operate the swing hydraulic motor 2A and the arm cylinder 8, and the right operating lever is used, for example, to operate the boom cylinder 7 and the bucket cylinder 9. The traveling lever includes a left traveling lever and a right traveling lever. The left travel lever is used, for example, to operate the left travel hydraulic motor 2ML, and the right travel lever is used, for example, to operate the right travel hydraulic motor 2MR. The left travel lever may be configured to interlock with the left travel pedal. The same applies to the right traveling lever.

The operating device 26 is provided with a switch NS. In this embodiment, the switch NS is a push button switch. The operator can manually operate the operating device while pressing the switch NS with his / her finger. The switch NS is used, for example, when executing a machine control function.

The shuttle valve 32 has two inlet ports and one outlet port, and outputs (outflows) the hydraulic oil having the higher pressure among the hydraulic oils input (inflow) to each of the two inlet ports. ). In the shuttle valve 32, one of the two inlet ports is connected to the operating device 26 and the other is connected to the proportional valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of the corresponding control valve in the control valve unit 17 through a pilot line. Therefore, the shuttle valve 32 can make the higher of the pilot pressure generated by the operating device 26 and the pilot pressure generated by the proportional valve 31 act on the pilot port of the corresponding control valve. That is, the excavator controller 30 causes the pilot pressure generated by the proportional valve 31, which is higher than the pilot pressure generated by the operating device 26, to act on the pilot port of the corresponding control valve, so that the operating device 26 is operated by the operator. Regardless of the operation, the corresponding control valve can be controlled to operate the corresponding operated unit. That is, the excavator controller 30 can autonomously operate the excavator 100.

The operating device 26 may be an electric type that outputs an electric signal instead of a hydraulic pilot type that generates a pilot pressure. In this case, the electric signal from the operating device 26 is input to the excavator controller 30. The excavator controller 30 can operate the hydraulic actuator according to the operation content for the operating device 26 by controlling the control valve in the control valve unit 17 according to the input electric signal. The control valve in the control valve unit 17 may be an electromagnetic solenoid type spool valve driven by a command from the shovel controller 30. Alternatively, an electromagnetic valve that operates in response to an electric signal from the shovel controller 30 may be arranged between the pilot pump 15 and each pilot port of the control valve in the control valve unit 17. In this case, when a manual operation using the electric operation device 26 is performed, the excavator controller 30 may operate the corresponding solenoid valve by the electric signal corresponding to the operation amount (for example, the lever operation amount). can. Specifically, the excavator controller 30 increases or decreases the pilot pressure generated by using the hydraulic oil discharged from the pilot pump 15 by the solenoid valve, thereby adjusting the corresponding control valve according to the operation amount of the operation device 26. Can be operated.

If the excavator 100 is an unmanned excavator, the operating device 26 may be omitted. Alternatively, when the excavator 100 is a remote-controlled excavator, the operating device 26 may be installed outside the excavator 100 (cabin 10).

In the present embodiment, the control system of the excavator 100 includes a discharge pressure sensor 28, an operating pressure sensor 29, an excavator controller 30, a proportional valve 31, a display device 40, an input device 42, a sound output device 43, a storage device 47, and a boom angle sensor. Includes S1, arm angle sensor S2, bucket angle sensor S3, body tilt sensor S4, turning state sensor S5, space recognition device S6, positioning device P1, and communication device T1.

The shovel controller 30, which is an example of the control device, is provided in the cabin 10, for example, and is configured to control the drive of the shovel 100. The excavator controller 30 may be realized by hardware, software, or a combination thereof. In the present embodiment, the excavator controller 30 is composed of a computer including a CPU, a volatile storage device, a non-volatile storage device, various input / output interfaces, and the like. Then, the excavator controller 30 realizes various functions by executing various programs stored in the non-volatile storage device.

For example, the shovel controller 30 sets a target rotation speed of the engine 11 based on a work mode or the like preset by a predetermined operation of an operator or the like, and controls the engine 11 to rotate constantly. Further, for example, the excavator controller 30 outputs a control command to the regulator 13 as necessary to change the discharge amount of the main pump 14. Further, for example, the excavator controller 30 may execute a machine guidance function for guiding the manual operation of the excavator 100 through the operating device 26 by the operator. Further, the excavator controller 30 may, for example, automatically support the manual operation of the excavator 100 through the operating device 26 by the operator, or may execute a machine control function for operating the excavator 100 independently of the manual operation. .. Alternatively, the machine control function may be a function for operating an unmanned excavator.

A part of the function of the excavator controller 30 may be realized by another controller (control device). That is, the above-mentioned function may be realized in a mode in which distributed processing is performed by a plurality of controllers. For example, the machine guidance function and the machine control function may be realized by a dedicated controller (control device).

In the present embodiment, the excavator controller 30 is configured to estimate at least one of the position, orientation, and size of the dump truck 200 (hereinafter referred to as “position and the like”). The details of the estimation of the position of the dump truck 200 by the excavator controller 30 will be described later.

The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. The discharge pressure sensor 28 transmits a detection signal regarding the discharge pressure to the excavator controller 30.

The operation pressure sensor 29, which is an example of the operation state detection device, is configured to detect the operation state of the operation device 26. In the present embodiment, the operating pressure sensor 29 detects the pilot pressure generated by the operating device 26. The operating pressure sensor 29 transmits a detection signal regarding the operating pressure to the excavator controller 30.

The operation state detection device may be an encoder, a potentiometer, a tilt sensor, or the like that can detect the tilt direction and tilt angle of the operation lever.

The proportional valve 31 is provided in a pilot line connecting the pilot pump 15 and the shuttle valve 32, and is configured so that the flow path area (cross-sectional area through which hydraulic oil can flow) can be changed. Then, the proportional valve 31 operates in response to a control command input from the shovel controller 30. With this configuration, the excavator controller 30 can, for example, supply the hydraulic oil discharged by the pilot pump 15 through the proportional valve 31 and the shuttle valve 32 in the control valve unit 17 even when the operating lever is not operated. It can be supplied to the pilot port of the corresponding control valve and the hydraulic actuator can be operated.

The display device 40 is configured to be able to display various images. In the present embodiment, the display device 40 is provided at a position easily visible to the operator seated in the driver's seat in the cabin 10, and displays various images under the control of the excavator controller 30. The display device 40 may be connected to the excavator controller 30 via an in-vehicle communication network such as CAN, or may be connected to the excavator controller 30 via a one-to-one dedicated line.

In the present embodiment, the display device 40 operates by receiving electric power from the storage battery 70. The storage battery 70 is charged with the electric power generated by the alternator 11a of the engine 11. The electric power of the storage battery 70 is also supplied to the shovel's electrical components 72 and the like other than the shovel controller 30 and the display device 40. Further, the starter 11b of the engine 11 is driven by the electric power from the storage battery 70 to start the engine 11.

The engine control device 74 is configured to control the engine 11. From the engine control device 74, various data indicating the state of the engine 11 (for example, data indicating the cooling water temperature detected by the water temperature sensor 11c) are transmitted to the excavator controller 30. The excavator controller 30 can store this data in an internal temporary storage unit (memory) and appropriately transmit it to the display device 40.

The input device 42 is a device for inputting information to the control device. In the present embodiment, the input device 42 is provided within reach from the operator seated in the driver's seat in the cabin 10, receives input from the operator, and outputs a signal corresponding to the input to the excavator controller 30. The input device 42 may be a touch panel, a knob switch provided at the tip of an operation lever, a button switch installed around the display device 40, a lever, a toggle, a rotary dial, or the like. The input device 42 transmits a signal related to the input to the excavator controller 30.

The sound output device 43 is configured to output sound. In the present embodiment, the sound output device 43 is provided in the cabin 10 and is connected to the excavator controller 30. The sound output device 43 is configured to output sound under the control of the shovel controller 30. Specifically, the sound output device 43 is a speaker, a buzzer, or the like. The sound output device 43 audibly outputs various information in response to an output command from the excavator controller 30.

The storage device 47 is configured to store information. In the present embodiment, the storage device 47 is provided in the cabin 10 and is configured to store various information under the control of the shovel controller 30. Specifically, the storage device 47 is a non-volatile storage medium such as a semiconductor memory. The storage device 47 may store information output by various devices during the operation of the excavator 100, or may store information acquired through the various devices before the operation of the excavator 100 is started. The storage device 47 may store data regarding a target construction surface acquired via, for example, a communication device T1 or the like, or set through an input device 42 or the like. The target construction surface may be set by the operator of the excavator 100, or may be set by the construction manager or the like.

The boom angle sensor S1 is attached to the boom 4 and is configured to detect the rotation angle (hereinafter referred to as “boom angle”) of the boom 4 with respect to the upper swing body 3. The boom angle sensor S1 transmits a detection signal regarding the boom angle to the excavator controller 30.

The arm angle sensor S2 is attached to the arm 5 and is configured to detect the rotation angle of the arm 5 with respect to the boom 4 (hereinafter referred to as “arm angle”). The arm angle sensor S2 transmits a detection signal regarding the arm angle to the excavator controller 30.

The bucket angle sensor S3 is attached to the bucket 6 and is configured to detect the rotation angle (hereinafter referred to as “bucket angle”) of the bucket 6 with respect to the arm 5. The bucket angle sensor S3 transmits a detection signal regarding the bucket angle to the excavator controller 30.

The boom angle sensor S1 is composed of, for example, at least one of a rotary encoder, an acceleration sensor, a gyro sensor, an inertial measurement unit, and the like. The boom angle sensor S1 may be a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of the hydraulic cylinder (boom cylinder 7) corresponding to the boom angle, or the like. The same applies to the arm angle sensor S2 and the bucket angle sensor S3.

The aircraft tilt sensor S4 is configured to detect the tilted state of the aircraft with respect to a predetermined virtual plane. The predetermined virtual plane is, for example, a virtual horizontal plane. The airframe is, for example, the upper swivel body 3. The machine body tilt sensor S4 is attached to the upper swivel body 3, for example, and has a roll angle which is a tilt angle around the front-rear axis of the upper swivel body 3 and a pitch angle which is a tilt angle around the left-right axis of the upper swivel body 3. To detect. The airframe tilt sensor S4 may be, for example, at least one such as an acceleration sensor, a gyro sensor, and an inertial measurement unit. Then, the machine body tilt sensor S4 transmits a detection signal regarding the tilt state to the excavator controller 30.

The turning state sensor S5 is configured to detect the turning state of the upper turning body 3. The turning state sensor S5 detects, for example, the turning angular velocity of the upper turning body 3. The turning state sensor S5 may detect the turning angle of the upper turning body 3. Specifically, the turning state sensor S5 is a gyro sensor, a resolver, a rotary encoder, or the like. The turning state sensor S5 transmits a detection signal regarding the turning angle or turning angular velocity of the upper turning body 3 toward the shovel controller 30.

The space recognition device S6 is configured to recognize the state of the space to be monitored. The state of space includes the existence or nonexistence of an object in that space. In the present embodiment, the space recognition device S6 is configured to recognize the state of the space around the shovel 100. Specifically, the space recognition device S6 displays the state of the space in front of the excavator 100, the sensor S6F that recognizes the state of the space in front of the excavator 100, the sensor S6L that recognizes the state of the space on the left side of the excavator 100, and the state of the space on the right side of the excavator 100. The sensor S6R for recognizing and the sensor S6B for recognizing the state of the space behind the excavator 100 are included.

The sensor S6F is attached to the front end of the upper surface of the cabin 10, the sensor S6L is attached to the left end of the upper surface of the upper swing body 3, the sensor S6R is attached to the right end of the upper surface of the upper swing body 3, and the sensor S6B is attached to the upper right end of the upper swing body 3. It is attached to the rear end of the upper surface of 3.

In this embodiment, the space recognition device S6 is a lidar. Specifically, the sensor S6F is composed of a combination of a plurality of lidars. The sensor S6F may be composed of one LIDAR. The same applies to the sensor S6B, the sensor S6L, and the sensor S6R. Further, each of the sensor S6B, the sensor S6F, the sensor S6L, and the sensor S6R may include a monocular camera. Further, the space recognition device S6 may be an RGB-D sensor, a stereo camera, a range image camera, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, a laser radar, or the like, and has an image pickup element such as a CCD or CMOS. , It may be a monocular camera to which a wide-angle sensor is attached. The space recognition device S6 transmits the acquired information to the excavator controller 30. The space recognition device S6 may transmit the acquired information to the display device 40.

The space recognition device S6 may function as an object detection device. In this case, the space recognition device S6 may detect an object existing around the excavator 100. The object to be detected is, for example, a person, an animal, a vehicle, a construction machine, a building, a hole, or the like. Further, the space recognition device S6 may calculate the distance between the space recognition device S6 or the excavator 100 and the detected object. In this case as well, the space recognition device S6 may function as an object detection device. In the case of an ultrasonic sensor, a millimeter wave radar, a laser radar, a lidar, an RGB-D sensor, an infrared sensor, or the like, the space recognition device S6 transmits a large number of signals (laser light, etc.) toward the surroundings. By receiving the reflected signal, the distance between the space recognition device S6 and the object and the direction in which the object exists may be detected.

In this embodiment, the space recognition device S6 is directly connected to the excavator controller 30. However, the space recognition device S6 may be connected to the display device 40 and indirectly connected to the excavator controller 30 via the display device 40.

The positioning device P1 is configured to measure the position of the upper swivel body 3. The positioning device P1 may be configured to detect the direction of the upper swivel body 3. In the present embodiment, the positioning device P1 is a GNSS compass, and derives the position and orientation of the upper swivel body 3. Then, the positioning device P1 transmits information regarding the position and orientation of the upper swivel body 3 toward the shovel controller 30. The orientation of the upper swing body 3 may be detected based on the output of the directional sensor attached to the upper swing body 3.

The communication device T1 is configured to communicate with an external device through a communication network including at least one such as a mobile communication network, a satellite communication network, a short-range wireless communication network, and an Internet network. The communication device T1 is, for example, a mobile communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G, and short-range wireless communication corresponding to a short-range wireless communication standard such as Wi-Fi or Bluetooth (registered trademark). A module, a satellite communication module for connecting to a satellite communication network, or the like.

Next, the details of the construction support system SYS will be described with reference to FIG. The construction support system SYS mainly includes a shovel 100, a dump truck 200, a management device 300, a support device 400, and a monitoring device 500. The excavator 100 (excavator controller 30), dump truck 200 (dump controller 210), management device 300 (control device 310), support device 400 (control device 410), and monitoring device 500 (control device 510) have a communication network 900. It is configured to be able to communicate via.

The dump truck 200 is an example of a transport vehicle. In the present embodiment, the dump truck 200 is a vehicle for carrying a load loaded by the excavator 100, and mainly has a dump controller 210, a display device 220, a communication device 230, and a positioning device 240. The load is, for example, soil, gravel, crushed stone, asphalt, concrete, etc. (hereinafter referred to as "earth and sand").

The dump controller 210 is composed of a computer including a CPU, a volatile storage device, a non-volatile storage device, various input / output interfaces, and the like. The dump controller 210 realizes various functions by executing various programs stored in the non-volatile storage device, for example. The dump controller 210 is configured to control the display device 220, the communication device 230, and the positioning device 240.

In the present embodiment, the dump controller 210 has an image generation unit. The image generation unit is configured to generate an image including an image relating to the target stop position (described later). Specifically, the image generation unit receives information regarding the target stop position calculated by the excavator controller 30 via the communication device 230. Then, the image generation unit generates an image that allows the driver of the dump truck 200 to stop the dump truck 200 at the target stop position, and displays the image on the display device 220.

The display device 220 is configured to display an image. In the present embodiment, the display device 220 is provided at a position easily visible to the driver seated in the driver's seat, and displays various images under the control of the dump controller 210. The display device 220 may be connected to the dump controller 210 via an in-vehicle communication network such as CAN, or may be connected to the dump controller 210 via a one-to-one dedicated line.

The communication device 230 is configured to communicate with an external device through a communication network including at least one such as a mobile communication network, a satellite communication network, a short-range wireless communication network, and an Internet network. The communication device 230 is, for example, a mobile communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G, a short-range wireless communication module corresponding to a short-range wireless communication standard such as Wi-Fi or Bluetooth, or a short-range wireless communication module. It is a satellite communication module for connecting to a satellite communication network.

The positioning device 240 is configured to measure the position of the dump truck 200. The positioning device 240 may be configured to detect the orientation of the dump truck 200. In this embodiment, the positioning device 240 is a GNSS compass and derives the position and orientation of the dump truck 200. Then, the positioning device 240 transmits information regarding the position and orientation of the dump truck 200 to the dump controller 210. The orientation of the dump truck 200 may be detected based on the output of the orientation sensor attached to the dump truck 200.

The management device 300 is a terminal device installed outside the construction site. In the present embodiment, the management device 300 is a stationary computer installed in a management center or the like outside the construction site, and mainly has a control device 310, a display device 320, and a communication device 330. However, the management device 300 may be a mobile terminal such as a notebook PC, a tablet PC, or a smartphone.

The control device 310 includes a computer including a CPU, a volatile storage device, a non-volatile storage device, various input / output interfaces, and the like. The control device 310 realizes various functions by executing various programs stored in the non-volatile storage device, for example. The control device 310 is configured to control the display device 320 and the communication device 330.

The display device 320 is configured to display an image. In the present embodiment, the display device 320 displays various images under the control of the control device 310.

The communication device 330 is configured to communicate with an external device through a communication network including at least one such as a mobile communication network, a satellite communication network, a short-range wireless communication network, and an Internet network. In the present embodiment, the communication device 330 is a mobile communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G, and a short-range wireless communication module corresponding to a short-range wireless communication standard such as Wi-Fi or Bluetooth. Or, a satellite communication module or the like for connecting to a satellite communication network.

The support device 400 is a mobile terminal, for example, a notebook PC, a tablet PC, a smartphone, or the like carried by a worker or the like at a construction site. In the present embodiment, the support device 400 is a smartphone carried by the driver of the dump truck 200, and mainly has a control device 410, a display device 420, a communication device 430, and a positioning device 440. The control device 410, the display device 420, and the communication device 430 are the same as the control device 310, the display device 320, and the communication device 330 in the management device 300.

In the present embodiment, the control device 410 has an image generation unit like the dump controller 210. The image generation unit is configured to generate an image including an image relating to the target stop position (described later). Specifically, the image generation unit receives information regarding the target stop position calculated by the excavator controller 30 via the communication device 430. Then, the image generation unit generates an image that allows the driver of the dump truck 200 to stop the dump truck 200 at the target stop position, and displays the image on the display device 420.

The positioning device 440 is configured to measure the position of the support device 400. The positioning device 440 may be configured to detect the orientation of the dump truck 200 when the support device 400 is attached to a cradle provided near the driver's seat of the dump truck 200. In this case, the positioning device 440 derives the direction of the dump truck 200 based on the output of the directional sensor attached to the support device 400, for example. Then, the positioning device 440 transmits information regarding the position and orientation of the dump truck 200 to the control device 410.

The monitoring device 500 is configured to monitor a predetermined area. In the present embodiment, the monitoring device 500 is mounted on a flying object such as a multicopter (drone) or an airship, and is configured to be able to monitor the construction site from the sky. Specifically, the monitoring device 500 mainly includes a control device 510, a communication device 530, a positioning device 540, and a space recognition device 550. The control device 510 and the communication device 530 are the same as the control device 310 and the communication device 330 in the management device 300. The monitoring device 500 may be attached to the upper end of a high structure such as a steel tower installed at a construction site, and the periphery of the structure may be monitored from the sky.

The positioning device 540 is configured to measure the position of the monitoring device 500. The positioning device 540 may be configured to detect the orientation of the monitoring device 500. In this embodiment, the positioning device 540 is a GNSS compass, and the position and orientation of the monitoring device 500 are derived. Then, the positioning device 540 transmits information regarding the position and orientation of the monitoring device 500 toward the control device 510. The orientation of the monitoring device 500 may be detected based on the output of the directional sensor attached to the monitoring device 500.

Like the space recognition device S6, the space recognition device 550 is configured to recognize the state of the space to be monitored. In the present embodiment, the space recognition device 550 is configured to image the construction site from the sky. Specifically, the space recognition device 550 is, for example, a monocular camera, a stereo camera, a range image camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, a laser radar, a LIDAR, an RGB-D sensor, an infrared sensor, or the like. .. In this embodiment, the space recognition device 550 is a monocular camera attached to the multicopter. The space recognition device 550 may function as an object detection device in the same manner as the space recognition device S6. In this case, the space recognition device 550 may detect an object existing in the construction site.

Next, with reference to FIGS. 4 and 5, a process for the shovel controller 30 to estimate the position of the dump truck 200 and the like (hereinafter referred to as “estimation process”) will be described. FIG. 4 is a flowchart showing an example of the estimation process. The excavator controller 30 repeatedly executes this estimation process in a predetermined control cycle. FIG. 5 shows an example of data used in the estimation process.

First, the excavator controller 30 extracts a point cloud related to the dump truck 200 (step ST1). In the present embodiment, the excavator controller 30 extracts the point cloud data related to the dump truck 200 from the point cloud data output by the lidar as the sensor S6F. The sensor S6F may be composed of a plurality of adjacent lidars.

The point cloud data is an example of three-dimensional data. In this embodiment, the point cloud data is represented by a distance image. A distance image is an image in which the distance from a predetermined point is used as the value of each element of the two-dimensional array.

For example, the excavator controller 30 extracts the point cloud data related to the dump truck 200 by removing the point cloud data related to an object other than the dump truck 200 such as the ground or a material.

For example, the excavator controller 30 may extract the difference between the distance image acquired at the first time point and the distance image acquired at the second time point after the first time point as point cloud data related to the dump truck 200. .. That is, the point cloud data relating to the object moved within the time between the first time point and the second time point may be extracted as the point cloud data relating to the dump truck 200. Alternatively, the excavator controller 30 may extract point cloud data whose height from the ground is equal to or higher than a predetermined height as point cloud data related to the dump truck 200. Alternatively, the excavator controller 30 may extract as point cloud data regarding the dump truck 200 by using any other method.

FIG. 5A shows an image 200G of a dump truck 200 captured by a monocular camera attached to LIDAR as a space recognition device S6. FIG. 5B shows the point cloud data PG1 regarding the dump truck 200 extracted by the excavator controller 30. The point cloud data PG1 shown in FIG. 5B corresponds to the image 200G of the dump truck 200 shown in FIG. 5A. The image 200G of the dump truck 200 captured by the monocular camera is not essential data for performing the estimation process. Therefore, the shooting of the dump truck 200 by the monocular camera may be omitted, or the monocular camera itself may be omitted.

After that, the excavator controller 30 creates a bottom removal process of the loading platform and a histogram of the normal (step ST2). In the present embodiment, the excavator controller 30 obtains the normal line a at the point of interest from the point of interest in the point cloud and its neighboring points by principal component analysis. That is, the excavator controller 30 obtains the normal line a at each point of interest constituting the point cloud relating to the five surfaces constituting the loading platform of the dump truck 200. The five surfaces are the bottom surface, the front surface (front panel), the rear surface (rear gate), the left side surface (left side gate), and the right side surface (right side gate). Then, the excavator controller 30 obtains the angle θ formed between the normal line a and the vertically upward normal line b by using the following equation (1).

ダンプトラック２００の荷台の形状は、略直方体であるため、ショベルコントローラ３０は、ダンプトラック２００の荷台の前面、後面、左側面、右側面、及び底面の５つの面に対して角度θを算出できる。ダンプトラック２００の荷台の底面は、荷台の側面とは異なり、ダンプトラック２００の傾き、空間認識装置Ｓ６とダンプトラック２００との間の距離、又は、空間認識装置Ｓ６から見たダンプトラック２００の向き等の変化に応じて死角となる部分の範囲が変化し易い。そのため、本実施形態では、ショベルコントローラ３０は、荷台の底面に関する点群データを除去した上で、残りの点群データに基づいてダンプトラック２００の位置等を推定するように構成されている。

Since the shape of the loading platform of the dump truck 200 is a substantially rectangular parallelepiped, the excavator controller 30 can calculate the angle θ with respect to the five surfaces of the front surface, the rear surface, the left side surface, the right side surface, and the bottom surface of the loading platform of the dump truck 200. .. The bottom surface of the loading platform of the dump truck 200 is different from the side surface of the loading platform, and the inclination of the dump truck 200, the distance between the space recognition device S6 and the dump truck 200, or the orientation of the dump truck 200 as seen from the space recognition device S6. The range of the blind spot is likely to change according to changes such as. Therefore, in the present embodiment, the excavator controller 30 is configured to remove the point cloud data relating to the bottom surface of the loading platform and then estimate the position of the dump truck 200 or the like based on the remaining point cloud data.

When the dump truck 200 is located on a horizontal plane, the normal direction of the bottom surface of the loading platform is vertically upward. Then, the excavator controller 30 can determine that the normal line a has substantially the same direction as the vertically upward normal line b if the angle θ obtained by the equation (1) is equal to or less than a predetermined value, and the point of interest at that time is It can be determined that it is one of the points related to the bottom surface of the loading platform. Therefore, the excavator controller 30 can remove the point cloud data related to the bottom surface of the loading platform from the point cloud data related to the dump truck 200 by obtaining the angle θ. In this way, the excavator controller 30 can extract the surfaces whose angle θ obtained by the equation (1) is larger than the predetermined value as the remaining four surfaces of the loading platform.

FIG. 5C shows the point cloud data PG2 obtained by removing the point cloud data relating to the bottom surface of the loading platform from the point cloud data PG1 relating to the dump truck 200. Specifically, FIG. 5C shows point cloud data PG2 relating to the four surfaces constituting the loading platform of the dump truck 200. The four surfaces are the front surface (front panel), the rear surface (rear gate), the left side surface (left side gate), and the right side surface (right side gate).

Then, the excavator controller 30 takes the z component, which is a component in the depth direction (front-back direction of the excavator 100) of the normal line, on the horizontal axis based on the point group data regarding the remaining four surfaces of the loading platform, and has the normal line with respect to the depth direction. Create a two-dimensional histogram with the angle on the vertical axis. In the present embodiment, the point cloud data at this time excludes the point cloud data relating to the bottom surface of the loading platform.

The point cloud data includes an error due to noise, and the normal obtained from the point cloud data also includes an error due to noise. Therefore, in the present embodiment, the excavator controller 30 assumes that the variation in the error of the point group follows a Gaussian distribution, and that the two-dimensional histogram created from the normal follows a mixed Gaussian distribution. Then, the excavator controller 30 estimates the normal of each surface by performing point cloud clustering using an EM (Expectation-Maximization) algorithm. For example, the excavator controller 30 uses a normal that corresponds to a class including the largest number of point clouds as a representative normal, and clusters the point cloud belonging to that class with a three-dimensional distance to a neighboring point as a threshold.

After that, the excavator controller 30 detects the surface (step ST3). In the present embodiment, the excavator controller 30 detects the point cloud related to the dump truck 200 after the point cloud related to the bottom surface of the loading platform is removed as a surface. Specifically, the excavator controller 30 detects at least one of the four surfaces constituting the loading platform of the dump truck 200.

More specifically, the excavator controller 30 detects a point cloud belonging to a class in which the number of points constituting the clustered point cloud as described above is equal to or more than a predetermined value and the three-dimensional distance is the smallest as a surface. do.

Further, in order to determine whether the detected surface corresponds to the short side (front or rear surface of the loading platform) or the long side (left side surface or right surface surface of the loading platform) of the rectangle, the excavator controller 30 determines whether the detected surface corresponds to the detected surface. Extract the normals whose angle to the normals is 90 degrees. Then, as described above, the excavator controller 30 detects another surface by clustering using the three-dimensional distance to the neighboring point. Then, the excavator controller 30 determines whether each surface corresponds to the short side or the long side of the rectangle based on the detected length of each surface.

After that, the excavator controller 30 estimates the normal (step ST4). In the present embodiment, the excavator controller 30 performs normal estimation by RANSAC (Random Sample Consensus). Specifically, the shovel controller 30 distinguishes the surface corresponding to the short side (or long side) from the surface corresponding to the long side (or short side) with respect to the surface obtained from the representative normal. Perform normal estimation by RANSAC.

Since the point cloud with respect to the detected surface typically contains a lot of noise, the direction of the derived normal also contains an error. Therefore, in the present embodiment, the excavator controller 30 is configured to reduce the error due to the influence of noise by executing the estimation of the normal by RANSAC on the representative surface.

After that, the excavator controller 30 estimates the position of the dump truck 200 and the like (step ST5). In the present embodiment, the excavator controller 30 detects a surface constituting the loading platform from the direction of the normal and the center of gravity of the surface obtained as described above, and the dump truck 200 is based on the position and orientation of the detected surface. Estimate the position etc. Since the shape of the loading platform of the dump truck 200 is a substantially rectangular parallelepiped, the excavator controller 30 can estimate the position and orientation of the dump truck 200 by obtaining the shape of the surface constituting the substantially rectangular parallelepiped.

FIG. 5D shows a rectangular RS representing the right side surface of the loading platform of the dump truck 200, which is one of the surfaces detected by the excavator controller 30. Specifically, the rectangular RS corresponds to the inner surface of the right side gate. In this case, the excavator controller 30 can estimate the size (tonnage, vehicle type, etc.) of the dump truck 200 from at least one three-dimensional distance of the four sides of the rectangular RS, for example. In this case, the excavator controller 30 derives the size of the dump truck 200 by referring to a table that systematically stores the relationship between the three-dimensional distance of at least one of the four sides of the rectangular RS and the size of the dump truck 200. You may. For example, the excavator controller 30 refers to the table using the three-dimensional distance (corresponding to the length of the long side of the right side gate) of the long side of the rectangle RS representing the right side surface of the loading platform of the dump truck 200 as a search key and inputs it in advance. The vehicle type of the dump truck 200 equipped with the right side gate of such a length may be specified from the vehicle type information provided. Then, the excavator controller 30 may derive the size of the dump truck 200 (for example, the maximum load capacity, the vehicle length, the vehicle width, the vehicle height, etc.) stored in association with the vehicle type of the specified dump truck 200. Further, the excavator controller 30 can estimate the direction of the dump truck 200 from the direction of the normal of the rectangular RS, for example. Further, the excavator controller 30 can estimate the position of the dump track 200 from, for example, the output of the positioning device P1 and the three-dimensional distance between at least one of the four vertices of the rectangular RS and the space recognition device S6. ..

The excavator controller 30 may be configured to estimate at least one of the position, orientation, and size of the dump truck 200 based on information about the detected two or more surfaces. For example, the excavator controller 30 may derive a rectangle representing the rear surface of the dump truck 200 in addition to the rectangular RS representing the right side surface of the dump truck 200 carrier. The rectangle representing the rear surface of the dump truck 200 corresponds to, for example, the outer surface of the rear gate. In this case, the excavator controller 30 is, for example, the position of the dump truck 200 tentatively estimated from the rectangle RS representing the right side surface of the loading platform, the position of the dump truck 200 tentatively estimated from the rectangle representing the rear surface of the loading platform, and the like. Based on the above, the position of the dump truck 200 and the like may be finally estimated. Alternatively, the excavator controller 30 estimates a part (for example, only the orientation) of the position of the dump truck 200 from the rectangle RS representing the right side surface of the loading platform, and the position of the dump truck 200 etc. from the rectangle representing the rear surface of the loading platform. The rest (eg position and size) may be estimated.

By this estimation process, the excavator controller 30 can accurately estimate the position of the dump truck 200 and the like. Therefore, the excavator controller 30 can appropriately and autonomously operate the hydraulic actuator of the excavator 100, for example, when executing the machine control function. Specifically, the shovel controller 30 can improve the reliability of the loading work by the shovel 100 by appropriately and autonomously operating the hydraulic actuator mounted on the shovel 100. In the above embodiment, since the excavator controller 30 detects the dump truck 200 from the rear, the bottom surface of the loading platform is removed and the right side surface is extracted. However, when the dump truck 200 is detected from above, the bottom surface is removed. Extraction can improve the estimation accuracy of the position of the dump truck 200 and the like. In this case, the excavator controller 30 can remove the front surface, the rear surface, the left side surface, and the right side surface, and determine the position and orientation of the dump truck 200 based on the extracted bottom surface. In this way, the surface to be removed and the surface to be extracted are changed according to the direction in which the dump truck 200 is detected.

Next, with reference to FIG. 6, the effect of the estimation process will be described. FIG. 6 shows an example of a work site where earth and sand are loaded onto the dump truck 200 by the excavator 100. Specifically, FIG. 6 is a top view of the excavator 100 and the dump truck 200 at the construction site. In FIG. 6, the excavator 100 drawn by the solid line represents the state of the excavator 100 when the excavation operation is completed, and the excavator 100 drawn by the alternate long and short dash line represents the state of the excavator 100 before the excavation operation is started. .. In the present embodiment, the soil removal operation is a bucket opening operation or a combined operation including a bucket opening operation, and does not include a turning operation. Typically, the soil removal operation is a combined operation including a bucket opening operation, an arm opening operation, and a boom lowering operation. Further, the bucket 6A drawn by the solid line represents the state of the bucket 6 when the excavation operation is completed, and the bucket 6B drawn by the alternate long and short dash line represents the state of the bucket 6 before the soil removal operation is started. Further, the thick broken line in FIG. 6 represents a locus drawn by a predetermined point on the back surface of the bucket 6.

In the loading operation, the operator of the excavator 100 usually performs a boom raising and turning operation by using the operating device 26 after the excavation operation is completed. In the present embodiment, the operator performs a combined operation including a right turn operation. Specifically, the operator performs the boom raising operation and the right turning operation until the posture of the excavator 100 becomes the posture shown by the alternate long and short dash line, that is, until the predetermined point on the back surface of the bucket 6 reaches the point P2. Perform compound operations including and. The combined operation may include an opening / closing operation of the bucket 6. This is to move the bucket 6 onto the loading platform while preventing the loading platform of the dump truck 200 from coming into contact with the bucket 6. By performing such a series of operations, the operator can load earth and sand on the loading platform of the dump truck 200.

The rectangular CB represented by the thick dotted line represents the position of the loading platform of the dump truck 200 estimated by the excavator controller 30 by the estimation process. The two-dot chain line represents the direction of the dump truck 200 estimated by the excavator controller 30. Specifically, the alternate long and short dash line represents the front-rear axis L1 of the dump truck 200.

As shown in FIG. 6, the excavator controller 30 estimates the position of the dump truck 200 and the like so that, for example, the swivel axis SA of the excavator 100 and the front-rear axis L1 of the dump truck 200 do not intersect each other. Can be recognized. That is, the excavator controller 30 recognizes that no matter how the turning position of the upper swivel body 3 is changed, the front-rear axis of the upper swivel body 3 and the front-rear axis L1 of the dump truck 200 cannot be made parallel. can.

Then, when it is possible to recognize that the positional relationship is such, the excavator controller 30 does not properly rotate the upper swivel body 3 during the soil discharge operation, for example, the earth and sand taken in the bucket 6 is taken in. It can be recognized that the above can not be discharged evenly onto the loading platform of the dump truck 200.

If the front-rear axis of the upper swivel body 3 and the front-rear axis L1 of the dump truck 200 are parallel to each other when the soil removal operation is started, the excavator 100 is taken into the bucket 6 only by executing the soil removal operation. It is possible to evenly spread the earth and sand and the like on the loading platform of the dump truck 200. This is because the bucket 6 can be moved along the front-rear axis L1 of the dump truck 200. However, when the front-rear axis of the upper swivel body 3 and the front-rear axis L1 of the dump truck 200 are not parallel, the excavator 100 dumps the earth and sand taken in the bucket 6 only by executing the earth removal operation. It cannot be evenly distributed on the loading platform of the truck 200. This is because the bucket 6 cannot be moved along the front-rear axis L1 of the dump truck 200 only by operating the excavation attachment AT. Therefore, when the shovel controller 30 recognizes that the swivel shaft SA of the shovel 100 and the front-rear shaft L1 of the dump truck 200 do not intersect each other, the shovel controller 30 autonomously swivels the upper swivel body 3 during the soil removal operation. By doing so, the earth and sand taken in the bucket 6 can be evenly distributed on the loading platform of the dump truck 200. That is, the operator of the excavator 100 can evenly spread the earth and sand taken in the bucket 6 onto the loading platform of the dump truck 200 without performing a turning operation during the earth removal operation.

Alternatively, when the shovel controller 30 recognizes that the swivel shaft SA of the shovel 100 and the front-rear shaft L1 of the dump truck 200 do not intersect with each other, the shovel controller 30 drives the traveling hydraulic motor 2M to operate the lower traveling body 1. You may. Specifically, the excavator controller 30 may move the excavator 100 so that the swivel axis SA of the excavator 100 and the front-rear axis L1 of the dump truck 200 intersect.

Alternatively, the excavator controller 30 may inform the operator of the excavator 100 that the swivel axis SA of the excavator 100 and the front-rear axis L1 of the dump truck 200 do not intersect each other. This is to encourage the operator to move the shovel 100 so that the turning shaft SA of the shovel 100 and the front-rear shaft L1 of the dump truck 200 intersect.

Alternatively, the excavator controller 30 may notify the driver of the dump truck 200 that the turning axis SA of the excavator 100 and the front-rear axis L1 of the dump truck 200 do not intersect each other. This is to encourage the driver to move the dump truck 200 so that the turning shaft SA of the excavator 100 and the front-rear shaft L1 of the dump truck 200 intersect. In this case, the excavator controller 30 may derive a target stop position and present information about the target stop position to the driver of the dump truck 200. The target stop position is a position where the dump truck 200 should be stopped, and is represented by, for example, a rectangle corresponding to the occupied area of the dump truck 200 when the dump truck 200 is viewed from above. Then, the target stop position is typically set so that the front-rear axis L1 of the dump truck 200 stopped at the target stop position intersects the turning axis SA of the excavator 100.

As described above, being able to accurately estimate the position and the like of the dump truck 200 can realize the efficiency of the loading work by the excavator 100. The excavator controller 30 is assigned to at least one of the operator of the excavator 100 and the driver of the dump truck 200 so that the loading work is performed in a state where the turning shaft SA of the excavator 100 and the front-rear axis L1 of the dump truck 200 intersect. This is because it can work.

Next, another example of the estimation process will be described with reference to FIG. 7. FIG. 7 is a flowchart showing another example of the estimation process. The excavator controller 30 repeatedly executes this estimation process in a predetermined control cycle. The estimation process shown in FIG. 7 is different from the estimation process shown in FIG. 4 in that it has step ST0, but is the same as the estimation process shown in FIG. 4 in that it has steps ST1 to ST5. Therefore, in the following, the explanation of the common part is omitted, and the difference part is explained in detail.

In the estimation process shown in FIG. 7, the excavator controller 30 determines whether or not the dump truck 200 is located within a predetermined working range (step ST0). In the present embodiment, the excavator controller 30 executes this determination task before extracting the point cloud related to the dump truck 200 using the output of the space recognition device S6. That is, the excavator controller 30 does not extract the point cloud related to the dump truck 200 until it is determined that the dump truck 200 is located within the predetermined working range. Even though the dump truck 200 is located away from the excavator 100 (a position unsuitable for loading work), it prevents unnecessary calculations such as accurately estimating the position of the dump truck 200. Because.

Specifically, the excavator controller 30 determines whether or not the dump truck 200 is located within a predetermined working range based on the image captured by the monocular camera attached to the space recognition device S6. The excavator controller 30 may determine the presence or absence of the dump truck 200 based on the output of a device other than a monocular camera such as an ultrasonic sensor or a millimeter wave radar. That is, if the excavator controller 30 can execute the determination task of step ST0 at a calculation cost lower than the calculation cost for executing the task after step ST1, the dump controller 30 dumps based on the output of any device including LIDAR. The existence or nonexistence of the truck 200 may be determined. At this stage, the excavator controller 30 only needs to be able to determine whether or not the dump truck 200 exists around the excavator 100, and even if the dump truck 200 exists around the excavator 100, the position of the dump truck 200 This is because it is only necessary to be able to roughly recognize. For example, the excavator controller 30 determines the existence or nonexistence of the dump truck 200 by pattern matching for collating the image of the dump truck 200 registered in advance with the image portion included in the image captured by the monocular camera, or the dump truck 200. You may recognize the position of. Alternatively, the excavator controller 30 may determine whether or not the image of the dump truck 200 exists in the image captured by the monocular camera by using deep learning, or may recognize the position of the dump truck 200. good. The excavator controller 30 recognizes, for example, the coordinates of the center of the loading platform of the dump truck 200, the coordinates of the rear end of the loading platform of the dump truck 200, or the like as the position of the dump truck 200. The coordinates are, for example, coordinates in a coordinate system centered on the position of the monocular camera.

Further, the excavator controller 30 may recognize the position and orientation of the dump truck 200 based on the image captured by the monocular camera. In this case as well, the excavator controller 30 only needs to be able to roughly recognize the position and orientation of the dump truck 200.

The predetermined work range is, for example, a range suitable for loading work. That is, the fact that the dump truck 200 is located within a predetermined work range means that the dump truck 200 is located within a range suitable for the loading work. In the present embodiment, the predetermined working range is set as a coordinate range in the coordinate system centered on the position of the monocular camera.

The excavator controller 30 determines whether or not the dump truck 200 is located within the predetermined work range based on the recognized position of the dump truck 200 and the set predetermined work range. The excavator controller 30 determines that the dump truck 200 is located within the predetermined work range, for example, when the coordinates of the center of the loading platform of the dump truck 200 are included in the coordinate range corresponding to the predetermined work range. do.

When it is determined that the dump truck 200 is not located within the predetermined work range (NO in step ST0), the excavator controller 30 ends the estimation process this time without executing the tasks after step ST1.

On the other hand, when it is determined that the dump truck 200 is located within the predetermined work range (YES in step ST0), the excavator controller 30 executes the tasks after step ST1 and estimates the position of the dump truck 200 and the like. ..

Further, once it is determined that the dump truck 200 is located within the predetermined work range, the shovel controller 30 omits the determination of step ST0 in the estimation process from the next time onward, and performs the task after step ST1. You may do it. For example, in a predetermined period after determining that the dump truck 200 is located within a predetermined working range, or until the estimation process is repeated a predetermined number of times, the excavator controller 30 omits the determination in step ST0. The task after step ST1 may be executed. This is to reduce the calculation load.

With this configuration, the excavator controller 30 can reduce the calculation load related to the estimation of the position of the dump truck 200 and the like. When the dump truck 200 does not exist around the excavator 100, or when the dump truck 200 is located in a place unsuitable for loading work, the excavator controller 30 estimates the position of the dump truck 200 and the like. This is because the task after step ST1 which is a task is not executed.

As described above, it has a lower traveling body 1 according to an embodiment of the present invention, an upper turning body 3 rotatably mounted on the lower traveling body 1, and an attachment attached to the upper turning body 3. The construction support system SYS, which is a system for the excavator 100, acquires three-dimensional data about the dump truck 200 based on the output of the object detection device, and is based on the information about at least one surface constituting the loading platform of the dump truck 200. It is configured to estimate at least one of the position, orientation, and size of the dump truck 200. The object detection device is typically a space recognition device S6 attached to the excavator 100. However, the object detection device may be a space recognition device installed outside the excavator 100. For example, the object detection device may be attached to another construction machine, a building, a utility pole, an air vehicle, or the like.

With this configuration, the construction support system SYS can more appropriately support the loading work by the excavator 100. This is because at least one of the position, orientation, and size of the dump truck 200 can be accurately estimated. Further, since the construction support system SYS can estimate at least one of the position, orientation, and size of the dump truck 200 without performing image recognition processing using machine learning such as deep learning, a trained model or the like can be used. Does not need to be prepared and can be easily introduced.

The construction support system SYS may be configured to extract information about at least one surface constituting the loading platform of the dump truck 200 from the three-dimensional data about the dump truck 200. The at least one surface constituting the loading platform of the dump truck 200 is typically at least one of the four sides constituting the loading platform of the dump truck 200.

The construction support system SYS may be configured to extract the three-dimensional data regarding the loading platform of the dump truck 200 from the three-dimensional data regarding the dump truck 200. For example, the construction support system SYS dumps by removing the three-dimensional data related to the earth and sand, the road surface, the tires of the dump truck 200, etc. around the dump truck 200 from the three-dimensional data output by the space recognition device S6. It may be configured to extract three-dimensional data about the truck bed.

The construction support system SYS may be configured to preferentially extract information on a surface whose overall shape can be recognized when extracting information on at least one surface constituting the loading platform of the dump truck 200. .. In this case, the surface on which the overall shape can be recognized may be, for example, a surface on which three or more corners can be recognized. Alternatively, the surface on which the overall shape can be recognized may be, for example, a surface on which two or more corners can be recognized. Typically, the surface on which the overall shape is recognizable is at least one of the four sides constituting the bed of the dump truck 200. When viewed from the space recognition device S6, the bottom surface of the loading platform of the dump truck 200 is often hidden behind the side surface constituting the loading platform of the dump truck 200. Therefore, in the above-described embodiment, the construction support system SYS is used. The bottom surface of the loading platform of the dump truck 200 is not preferentially extracted. However, in another embodiment, the construction support system SYS may preferentially extract the bottom surface of the loading platform of the dump truck 200.

With this configuration, the construction support system SYS can estimate at least one of the position, orientation, and size of the dump truck 200 based on the information about the surface on which the overall shape can be recognized, so that the reliability of the estimation result can be improved. Can be improved.

Information about at least one surface constituting the loading platform of the dump truck 200 is preferably information about a surface other than the bottom surface of the loading platform of the dump truck 200. As described above, the bottom surface of the loading platform of the dump truck 200 is often hidden behind the side surface constituting the loading platform of the dump truck 200 when viewed from the space recognition device S6.

The construction support system SYS is based on the information on the shape of at least one surface constituting the loading platform of the dump truck 200 among the information on at least one surface constituting the loading platform of the dump truck 200, or the size of the dump truck 200 or. It may be configured to derive the type. The information about the shape of the surface is, for example, the three-dimensional distance of the long side or the short side of the rectangle.

The construction support system SYS derives the position of the dump truck 200 based on the information on the position of at least one surface constituting the loading platform of the dump truck 200 among the information on at least one surface constituting the loading platform of the dump truck 200. It may be configured as follows. The information regarding the position of the surface is, for example, the coordinates of the center of gravity of the surface, the coordinates of a point on the contour of the surface, or the like.

The construction support system SYS derives the orientation of the dump truck 200 based on the information regarding the orientation of at least one surface constituting the loading platform of the dump truck 200 among the information regarding at least one surface constituting the loading platform of the dump truck 200. It may be configured as follows. The information regarding the orientation of the surface is, for example, the orientation of the normal of the surface, the extension direction of the long side or the short side of the surface, and the like.

The construction support system SYS may be configured to determine the existence or nonexistence of the dump truck 200 before acquiring the three-dimensional data regarding the dump truck 200. Alternatively, the construction support system SYS may be configured to estimate at least one of the position, orientation, and type of the dump truck 200 before acquiring the three-dimensional data about the dump truck 200. Alternatively, the construction support system SYS may be configured to determine whether or not the dump truck 200 has entered the predetermined work range before acquiring the three-dimensional data regarding the dump truck 200. For example, the construction support system SYS may be configured to determine the existence or nonexistence of the dump truck 200 based on the image captured by the camera mounted on the excavator 100. In this case, the camera may be a monocular camera or a stereo camera. This is to prevent the task of high calculation cost for accurately estimating the position and the like of the dump truck 200 from being executed even though the dump truck 200 is away from the excavator 100. With this configuration, the construction support system SYS can prevent the calculation load from increasing unnecessarily.

The preferred embodiment of the present invention has been described in detail above. However, the invention is not limited to the embodiments described above. Various modifications or substitutions can be applied to the above-described embodiments without departing from the scope of the present invention. Also, the features described separately can be combined as long as there is no technical conflict.

For example, in the above-described embodiment, the excavator controller 30 is configured to remove point cloud data regarding the bottom surface of the loading platform of the dump truck 200 in the estimation process. However, the excavator controller 30 may be configured to remove point cloud data relating to another surface other than the bottom surface of the loading platform of the dump truck 200. For example, in a situation where the space recognition device S6 looks down on the dump truck 200 from a high position, the range of the blind spot on the bottom surface of the loading platform of the dump truck 200 tends to be narrow. On the other hand, for example, on the rear surface of the loading platform of the dump truck 200, the range of the blind spot is likely to change according to the change in the distance between the space recognition device S6 and the dump truck 200. In this case, the excavator controller 30 may be configured to remove the point cloud data regarding the rear surface of the loading platform of the dump truck 200 in the estimation process.

1 ... Lower traveling body 1C ... Crawler 1CL ... Left crawler 1CR ... Right crawler 2 ... Swivel mechanism 2A ... Swivel hydraulic motor 2M ... Traveling hydraulic motor 2ML ... Left traveling Hydraulic motor 2MR ・ ・ ・ Right traveling hydraulic motor 3 ・ ・ ・ Upper swivel body 4 ・ ・ ・ Boom 5 ・ ・ ・ Arm 6 ・ ・ ・ Bucket 7 ・ ・ ・ Boom cylinder 8 ・ ・ ・ Arm cylinder 9 ・ ・ ・ Bucket cylinder 10 ... Cabin 11 ... Engine 11a ... Alternator 11b ... Starter 11c ... Water temperature sensor 13 ... Regulator 14 ... Main pump 14c ... Oil temperature sensor 15 ... Pilot pump 17 ... Control valve unit 26 ... Operating device 28 ... Discharge pressure sensor 29 ... Operating pressure sensor 30 ... Excavator controller 31 ... Proportional valve 32 ... Shuttle valve 40 ... Display Device 42 ・ ・ ・ Input device 43 ・ ・ ・ Sound output device 47 ・ ・ ・ Storage device 70 ・ ・ ・ Storage battery 72 ・ ・ ・ Electrical equipment 74 ・ ・ ・ Engine control device 75 ・ ・ ・ Dial 100 ・ ・ ・ Excavator 200 ・・ ・ Dump truck 210 ・ ・ ・ Dump controller 220 ・ ・ ・ Display device 230 ・ ・ ・ Communication device 240 ・ ・ ・ Positioning device 300 ・ ・ ・ Management device 310 ・ ・ ・ Control device 320 ・ ・ ・ Display device 330 ・ ・ ・Communication device 400 ... Support device 410 ... Control device 420 ... Display device 430 ... Communication device 440 ... Positioning device 500 ... Monitoring device 510 ... Control device 530 ... Communication device 540 ... Positioning device 550 ... Space recognition device 900 ... Communication network NS ... Switch P1 ... Positioning device S1 ... Boom angle sensor S2 ... Arm angle sensor S3 ... Bucket angle Sensor S4 ... Aircraft tilt sensor S5 ... Turning state sensor S6 ... Space recognition device S6B, S6F, S6L, S6R ... Sensor SYS ... Construction support system T1 ... Communication device

Claims (13)

With the lower running body,
The upper swivel body mounted on the lower traveling body so as to be swivel,
A system for excavators having an attachment attached to the upper swing body.
Three-dimensional data about the dump truck is acquired based on the output of the object detection device, and at least one of the position, orientation, and size of the dump truck is obtained based on the information about at least one surface constituting the loading platform of the dump truck. Estimate one,
A system for excavators. Extracting information about the at least one surface from the three-dimensional data about the dump truck.
The system for excavators according to claim 1. Extracting the three-dimensional data regarding the loading platform of the dump truck from the three-dimensional data regarding the dump truck.
The system for excavators according to claim 1 or 2. When extracting the information about the at least one surface, the information about the surface whose overall shape can be recognized is preferentially extracted.
The system for excavators according to claim 2. The surface on which the overall shape can be recognized is a surface on which three or more corners can be recognized.
The system for excavators according to claim 4. The information regarding the at least one surface is information regarding a surface other than the bottom surface of the loading platform of the dump truck.
The system for excavators according to any one of claims 1 to 5. The size or type of the dump truck is derived based on the information regarding the shape of the at least one surface among the information regarding the at least one surface.
The system for excavators according to any one of claims 1 to 6. The position of the dump truck is derived based on the information regarding the position of the at least one surface among the information regarding the at least one surface.
The system for excavators according to any one of claims 1 to 7. The orientation of the dump truck is derived based on the information regarding the orientation of the at least one surface of the information regarding the at least one surface.
The system for excavators according to any one of claims 1 to 8. The existence or nonexistence of the dump truck is determined before acquiring the three-dimensional data regarding the dump truck.
The system for excavators according to any one of claims 1 to 9. Estimate at least one of the position, orientation, and type of the dump truck before acquiring the three-dimensional data about the dump truck.
The system for excavators according to any one of claims 1 to 10. Determining whether or not the dump truck has entered a predetermined range,
The system for excavators according to any one of claims 1 to 11. The presence or absence of the dump truck is determined based on the image captured by the camera mounted on the excavator.
The system for excavators according to any one of claims 1 to 12.

Images