JP7501176B2 - Mobile Crane - Google Patents

Description

The present invention relates to a mobile crane.

Conventionally, when transporting cargo using a mobile crane, the operator of the mobile crane must operate multiple operating tools, such as a slewing operating tool for operating the boom of the crane, a lifting operating tool, an extension operating tool, and a winch operating tool for lifting and lowering cargo. The operator of the mobile crane must operate the multiple operating tools simultaneously to operate the mobile crane so that the cargo does not come into contact with structures at the work site. Depending on the situation at the work site, the operator of the mobile crane must transport the cargo within a range where the cargo and its surroundings cannot be seen from the operator's seat. With a mobile crane configured in this way, it may be difficult to transport the cargo to the destination depending on the operator's skill and the situation at the work site. Therefore, a crane that automatically transports the cargo based on the cargo's movement route input on a 3D map of the work site is known. For example, see Patent Document 1.

The crane described in Patent Document 1 extracts three-dimensional information of the suspended load (luggage) based on three-dimensional information of the work site acquired by a stereo camera or the like at the end of the boom. The crane also creates a plan view based on the three-dimensional information of the work site. Next, the crane acquires the horizontal movement path of the suspended load input by the operator on the plan view. Furthermore, the crane creates an expanded sectional view formed along the horizontal movement path based on the three-dimensional information of the work site. The crane identifies the three-dimensional movement path of the suspended load by acquiring the height movement path of the suspended load on the expanded sectional view. The crane automatically moves the suspended load along the identified movement path.

However, because the crane described in Patent Document 1 obtains three-dimensional information of the work site using a stereo camera mounted at the tip of the boom, there may be areas where the three-dimensional information cannot be obtained. Furthermore, the 3D map is not created from three-dimensional information of the work site obtained in real time. For this reason, when using the 3D map to identify the movement path of the suspended load, the operator of the crane needs to select a route with sufficient margin so that the suspended load can be moved even if the conditions of the work site change.

JP 2018-95370 A

The objective of the present invention is to provide a mobile crane that can easily transport cargo to a target location while keeping track of the situation at the work site in real time.

The problem that the present invention aims to solve is as described above, and the means for solving this problem will be explained next.

The mobile crane according to one embodiment of the present invention is a mobile crane with a boom that can be raised and lowered on a rotating base, and includes a camera that takes pictures of the mobile crane with the boom from above, a display device that displays images taken by the camera and allows input of the movement trajectory of the boom, and a control device that controls the actuator of the mobile crane.

The control device acquires an image including the mobile crane and the surroundings of the mobile crane captured by the imaging device, and displays the image on the display device. The control device calculates the coordinates of the boom's rotation center in the coordinate system of the image and the coordinates of the boom's tip in the coordinate system of the image from the image and the boom's attitude information. Furthermore, the control device calculates the coordinates in the coordinate system of the image of a passing point through which the boom's tip passes, which is set on the image. Next, the control device controls the actuator of the mobile crane so that the boom tip passes through the coordinates of the passing point at a predetermined speed.

In the above configuration, the control device for the mobile crane calculates the coordinates of the center of rotation of the mobile crane and the coordinates of the tip of the boom in an image captured by a camera device that includes the mobile crane and its surroundings. By calculating the coordinates of the center of rotation of the mobile crane and the coordinates of the tip of the boom, the control device can convert the coordinate system of the image into a crane coordinate system based on the center of rotation of the mobile crane.

Therefore, the operator of the mobile crane can set a passing point for the tip of the boom on the image while checking the situation around the mobile crane on the image in real time. The control device also performs automatic transport control that controls each actuator so that the tip of the boom passes through the passing point. This allows the operator to transport cargo to the destination with simple operations while grasping the situation at the work site in real time using the mobile crane.

From another perspective, it is preferable that the mobile crane of the present invention includes the following configuration: The control device detects an image of the tip of the boom from the image captured by the imaging device, and calculates the coordinates of the boom's center of rotation in the image coordinate system and the coordinates of the boom tip in the image coordinate system from the orientation and size of the image of the tip of the boom and the attitude information of the boom.

In the above configuration, the control device automatically detects the coordinates of the boom's rotation center and the coordinates of the boom's tip in the coordinate system of the image including the mobile crane captured by the imaging device, by comparing the coordinates with a previously acquired image of the boom's tip. The control device can easily calculate the coordinates of the passing point of the boom tip specified on the image based on the coordinates of the boom's rotation center. This allows the operator to easily transport cargo to the destination location using the mobile crane while grasping the situation at the work site in real time.

The mobile crane according to one embodiment of the present invention is a mobile crane with a boom that can be raised and lowered on a rotating base, and includes a camera that captures images of the mobile crane with the boom and the surroundings of the mobile crane from above, a first GNSS receiver that detects the absolute coordinates of the boom's rotation center, an orientation sensor that detects the orientation of the boom's extension direction, a second GNSS receiver that detects the absolute coordinates of the camera, a display device that displays images captured by the camera and allows input of the boom's movement trajectory, and a control device that controls the actuators of the mobile crane.

The control device acquires an image including the mobile crane and the surroundings of the mobile crane captured by the image capture device, and displays the image on the display device. Next, the control device acquires the absolute coordinates of the boom's rotation center from the first GNSS receiver, acquires the orientation of the boom's extension direction from the orientation sensor, and acquires the absolute coordinates of the image capture device from the second GNSS receiver. The control device calculates the absolute coordinates of the boom's tip from the absolute coordinates of the boom's rotation center, the absolute coordinates of the image capture device, and the boom's attitude information. The control device also calculates the absolute coordinates of a passing point through which the boom's tip, set at a position on the image, from the absolute coordinates of the boom's rotation center, the absolute coordinates of the image capture device, the absolute coordinates of the boom's tip, and the angle of view information of the image capture device, and controls the actuator of the mobile crane so that the boom tip passes through the absolute coordinates of the passing point at a predetermined speed.

In the above configuration, the mobile crane control device calculates the absolute coordinates of the boom's rotation center, the orientation of the boom's extension direction, the absolute coordinates of the camera device, the angle of view information of the camera device, and the boom's attitude information. The control device also calculates the coordinates of the rotation center and the coordinates of the boom's tip in the coordinate system of the image based on the absolute coordinates of the camera device and the angle of view information of the camera device. This makes it possible to convert the coordinate system of the image into an absolute coordinate system.

The operator of the mobile crane can therefore set a passing point for the tip of the boom on the image while checking the situation around the mobile crane on the image in real time. The control device also performs automatic transport control to control each actuator so that the tip of the boom passes through the passing point. This allows the operator to transport cargo to the destination with simple operations while grasping the situation at the work site in real time using the mobile crane.

From another perspective, it is preferable that the mobile crane of the present invention includes the following configuration: The imaging device is composed of an unmanned aerial vehicle and a camera provided on the unmanned aerial vehicle.

In the above configuration, the mobile crane can capture images of the mobile crane and its surroundings from any position using the imaging device. Therefore, the mobile crane can obtain images that correspond to the working range of the mobile crane and the situation at the work site. This allows the operator to use the crane to grasp the situation at the work site in real time, while easily transporting the cargo to the destination location.

From another perspective, it is preferable that the mobile crane of the present invention includes the following configuration. The control device calculates the vertical coordinate from the ground of a hook suspended from a wire rope that can be freely reeled in and out from the boom by a winch. When controlling the boom so that the tip of the boom passes through the passing point, the control device controls the winch so as to maintain the vertical coordinate of the hook from the ground.

In the above-mentioned configuration, the mobile crane maintains the vertical position of the load suspended from the hook even when the boom is raised or lowered so that the tip of the boom passes through the passing point specified by the operator. Therefore, the operator does not need to control the height of the load in accordance with the movement of the boom. This allows the operator to easily transport the load to the destination position while grasping the situation at the work site in real time using the mobile crane.

From another perspective, it is preferable that the mobile crane of the present invention includes the following configuration: When the control device acquires an operation signal of an operating tool for the winch, the control device controls the winch based on the operation signal.

In the above configuration, the mobile crane can switch from a state in which the vertical position of the load suspended from the hook is maintained during automatic transport control to a state in which the vertical position of the load can be changed by the operator's operation. The operator can concentrate on the operation of changing the vertical position of the load during automatic transport control. This allows the operator to transport the load to the desired location with simple operations while grasping the situation at the work site in real time using the mobile crane.

From another perspective, it is preferable that the mobile crane of the present invention includes the following configuration. The mobile crane further has an obstacle photography device composed of an unmanned aerial vehicle and a camera mounted on the unmanned aerial vehicle. In addition, the obstacle photography device photographs the area around the hook while following the hook.

In the above configuration, the obstacle photography device of the mobile crane photographs the area around the load. Therefore, the operator of the mobile crane can change the vertical position of the load while checking the area around the load. This allows the operator to easily transport the load to the destination while understanding the situation at the work site in real time using the mobile crane.

The terminology used in this specification is used for the purpose of defining only specific embodiments, and is not intended to limit the invention. Unless otherwise defined, all terms (including technical and scientific terms) used in this specification have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with the meaning in the context of the relevant technology and this disclosure, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

In the description of the present invention, it is understood that a number of techniques and processes are disclosed. Each of these has distinct advantages and may be used with one or more, or in some cases all, of the other disclosed techniques. Thus, for the sake of clarity, the description of the present invention refrains from unnecessarily repeating all possible combinations of the individual steps. However, the specification and claims should be read with the understanding that all such combinations are within the scope of the present invention.

In the following description, numerous specific examples are set forth to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be practiced without these specific examples. Therefore, the following disclosure should be considered as an example of the present invention, and is not intended to limit the present invention to the specific embodiments illustrated in the following drawings or description.

According to one embodiment of the present invention, the operator can use the crane to easily transport cargo to the destination while understanding the situation at the work site in real time.

1 is an overall view showing the overall configuration of a crane according to the present invention; FIG. 2 is a block diagram showing a control configuration of the crane according to the first embodiment of the present invention. FIG. 4 is a schematic diagram showing a state in which a positional relationship of a crane is acquired by a crane according to the first embodiment of the present invention. 1 is a block diagram showing a control configuration of an image processing apparatus according to a first embodiment of the present invention. FIG. 2 is a schematic diagram showing an image indicating a passing point of a crane according to the first embodiment of the present invention. FIG. 4 is a flowchart showing automatic transport control of a crane according to the first embodiment of the present invention. FIG. 11 is a block diagram showing a control configuration of a crane according to a second embodiment of the present invention. FIG. 11 is a block diagram showing a control configuration of an image processing apparatus according to a second embodiment of the present invention. FIG. 11 is a schematic diagram showing an image indicating a passing point of a crane according to a second embodiment of the present invention. FIG. 11 is a flowchart showing automatic transport control of a crane according to a second embodiment of the present invention. FIG. 11 is a block diagram showing a control configuration of an image processing apparatus according to another embodiment of the second embodiment of the present invention.

Below, using Figures 1 and 2, a mobile crane (rough terrain crane) according to a first embodiment of the present invention will be described as crane 1. Note that in this embodiment, a rough terrain crane will be described, but an all-terrain crane, truck crane, etc. may also be used.

As shown in Figures 1 and 2, the crane 1 is a mobile crane that can be moved to any location. The crane 1 has a vehicle 2, a crane device 6, a photographing device 20 (see Figure 2), a display device 24 (see Figure 2), a communication device 25 (see Figure 2), an image processing device 26 (see Figure 2), and a control device 27 (see Figure 2).

The vehicle 2 is a vehicle that transports the crane device 6. The vehicle 2 has multiple wheels 3 and runs using an engine 4 as a power source. The vehicle 2 is provided with outriggers 5. The outriggers 5 are composed of an extension beam that can be hydraulically extended on both sides in the width direction of the vehicle 2, and a hydraulic jack cylinder that can be extended in a direction perpendicular to the ground.

The crane device 6 is a work device that hoists cargo W with a wire rope. The crane device 6 includes a swivel base 7, a boom 9, a jib 9b, a main hook block 10, a sub-hook block 11, a hydraulic cylinder 12 for raising and lowering the load, a main winch 13, a main wire rope 14, a sub-winch 15, a sub-wire rope 16, and a cabin 17.

The swivel table 7 is a rotating device that allows the crane device 6 to rotate. The swivel table 7 is mounted on the frame of the vehicle 2 via annular bearings. The swivel table 7 is configured to be freely rotatable around the center of the annular bearing. The swivel table 7 is provided with a hydraulic swivel motor 8, which is an actuator. The swivel table 7 is configured to be swivelable in one direction and the other direction by the swivel hydraulic motor 8.

The boom 9 is a movable support that supports the main wire rope 14 and the sub wire rope 16 so that the load W can be lifted. The boom 9 is composed of multiple boom members. The base end of the base boom member of the boom 9 is swingably mounted at approximately the center of the swivel base 7. The boom 9 is configured to be freely extendable in the axial direction by moving each boom member with an actuator, an extension hydraulic cylinder 9a (see Figure 2). The boom 9 is also provided with a jib 9b.

The main hook block 10 and the sub-hook block 11 are used to suspend cargo W. The main hook block 10 is provided with multiple hook sheaves around which the main wire rope 14 is wound, and a main hook 10a for suspending cargo W. The sub-hook block 11 is provided with a sub-hook 11a for suspending cargo W.

The hoisting hydraulic cylinder 12 is an actuator that raises and lowers the boom 9 and maintains the position of the boom 9. The end of the cylinder section of the hoisting hydraulic cylinder 12 is connected to the swivel base 7 so that it can be raised and lowered, and the end of the rod section is connected to the base boom member of the boom 9 so that it can swing freely.

The main winch 13 and the sub winch 15 are used to wind up (hoist up) and unwind (lower down) the main wire rope 14 and the sub wire rope 16. The main winch 13 is configured so that the main drum around which the main wire rope 14 is wound is rotated by a main hydraulic motor (not shown) which serves as an actuator. The sub winch 15 is configured so that the sub drum around which the sub wire rope 16 is wound is rotated by a sub hydraulic motor (not shown) which serves as an actuator.

The cabin 17 is a housing that covers the driver's seat. The cabin 17 is mounted on the swivel base 7. A driver's seat (not shown) is provided. The driver's seat is provided with a driving operation tool 18 for operating the vehicle 2, a crane device operation tool 19 for operating the crane device 6, a remote control terminal for operating the imaging device 20, and the like (see FIG. 2).

As shown in Figures 2 and 3, the imaging device 20 captures images of the crane 1 and the surroundings of the crane 1 at the work site. The imaging device 20 includes an unmanned aerial vehicle 21 and a camera 22.

The unmanned aerial vehicle 21 is an autonomous unmanned aerial vehicle (drone) that can be remotely controlled and fly autonomously using a control signal. The unmanned aerial vehicle 21 is, for example, a multicopter with multiple propellers. The unmanned aerial vehicle 21 is configured to be able to send and receive control signals, etc., to and from the remote control terminal 23. This allows the unmanned aerial vehicle 21 to be moved to any position by the remote control terminal 23. The unmanned aerial vehicle 21 may be configured to follow a beacon attached to the boom 9 of the crane 1, an image of the boom 9 detected by the image processing device 26, an AR tag attached to the boom 9, etc.

The camera 22 is provided on the unmanned aerial vehicle 21. The camera 22 is configured to capture images vertically below the unmanned aerial vehicle 21. The camera 22 is configured to be able to acquire control signals and the like from the remote control terminal 23. The camera 22 is also configured to be able to transmit captured image data to the image processing device 26. This allows the camera 22 to start and stop capturing images, determine the capturing direction, zoom, and perform other operations using the remote control terminal 23.

The remote control terminal 23 is an operation terminal that operates the unmanned aerial vehicle 21 and the camera 22. The remote control terminal 23 is arranged in the cabin 17. The remote control terminal 23 is provided with an operation tool for the unmanned aerial vehicle 21 and an operation tool for the camera 22 in a portable housing. In other words, the remote control terminal 23 is configured as a portable operation terminal that can be carried outside the cabin 17. The remote control terminal 23 can place the unmanned aerial vehicle 21 above the crane 1 by operating the operation tool, and can hover at any position. The remote control terminal 23 is configured to be able to send and receive control signals to the unmanned aerial vehicle 21 and the camera 22. The remote control terminal 23 is also configured to be able to send and receive input signals, operation signals, etc. to the control device 27 of the crane 1. The remote control terminal 23 is provided with a display device 24.

The display device 24 displays the image P captured by the camera 22. The display device 24 is composed of a liquid crystal monitor or the like. The display device 24 is disposed in a position where it can be seen by the operator of the crane 1. The display device 24 is configured to be able to acquire the image P captured by the camera 22 via the remote control terminal 23. The display device 24 displays the image P captured by the camera 22 and information about the crane 1 calculated based on the image P captured by the camera 22. The display device 24 also displays an automatic transport control switch 24a for starting automatic transport control, and a winch operation switch 24b as a winch operation tool for the main winch 13 or sub winch 15.

The display device 24 is configured as an input device such as a touch panel that can input a control signal from the screen. The display device 24 is configured to be able to input the passing points of the tip of the boom 9, which is the movement trajectory of the boom 9, from the screen. The display device 24 may also be configured to be able to input operation signals for the unmanned aerial vehicle 21 and the camera 22 from the screen. The display device 24 may also be arranged in the cabin 17 independently from the remote control terminal 23.

The communication device 25 transmits and receives image data and control signals. The communication device 25 is provided in the remote control terminal 23. The communication device 25 can transmit and receive control signals for the unmanned aerial vehicle 21 and the camera 22 of the photographing device 20 to and from the image processing device 26. The communication device 25 can also transmit images P captured by the camera 22 to the image processing device 26. The communication device 25 can transmit and receive control signals between the unmanned aerial vehicle 21 and the camera 22. The communication device 25 can receive images P captured by the camera 22.

The image processing device 26 processes the image P (see FIG. 4) captured by the camera 22 of the image capture device 20. The image processing device 26 also controls the operation of the unmanned aerial vehicle 21 and the camera 22. The image processing device 26 may be configured in such a way that a CPU, ROM, RAM, HDD, etc. are connected via a bus, or may be configured as a one-chip LSI, etc. In this embodiment, the image processing device 26 is provided in the remote control terminal 23. The image processing device 26 stores various programs and data for controlling the operation of the unmanned aerial vehicle 21, the camera 22, etc. of the image capture device 20 and for processing image data. The image processing device 26 is connected to the control device 27. In other words, the image processing device 26 is considered to be electrically included in the control device 27. The image processing device 26 may be configured as one unit with the control device 27.

The image processing device 26 is connected to the display device 24. The image processing device 26 can acquire operation signals of the unmanned aerial vehicle 21 and the camera 22 from the display device 24. The image processing device 26 generates a control signal for the unmanned aerial vehicle 21 based on the operation signal for the unmanned aerial vehicle 21 input from the display device 24. Similarly, the image processing device 26 generates a control signal for the camera 22 based on the operation signal for the camera 22 input from the display device 24. The image processing device 26 is connected to the communication device 25. The image processing device 26 transmits the generated control signal to the unmanned aerial vehicle 21 or the camera 22 via the communication device 25.

The image processing device 26 detects the tip of the boom 9 from the image P captured by the camera 22 based on a learning model stored in advance. In this embodiment, the image processing device 26 uses a combination of vectorization using a color histogram of the image P, vectorization using an autoencoder, and vectorization using local features to calculate features by each vectorization. Vectorization using a color histogram is a method of vectorizing the color intensity and occurrence frequency as features. Vectorization using an autoencoder is a method of using the intermediate layer of the autoencoder as a feature representing the image P. The autoencoder is a three-layer neural network in which supervised learning is performed using the same data for the input layer and output layer. Vectorization using local features is a method of vectorizing the image P using KAZE local features.

The image processing device 26 calculates three feature amounts by vectorization of the image P using a color histogram, vectorization using an autoencoder, and vectorization using local features. The feature amounts may be the luminance values contained in the image P, an edge histogram, a correlation function between color and edges, etc. The image processing device 26 may also calculate local feature amounts of the image P, such as HOG (Histogram of Oriented Gradients) and SIFT (Scale Invariant Feature Transform).

Next, detection of the tip of the boom 9 and the extension direction of the boom 9 in the image processing device 26 of the crane 1 will be described using Figure 4. The image processing device 26 has a reference image holding unit 26a, a judgment image detection unit 26b, and a posture judgment unit 26c. The image processing device 26 has a learning model that has undergone supervised learning of a predetermined range of the tip of the boom 9. The image processing device 26 can detect the tip of the boom 9 and the extension direction of the boom 9 from three feature amounts of the image P captured by the camera 22.

As shown in FIG. 4, the reference image storage unit 26a is part of the image processing device 26, and stores a reference image Is of the tip portion of the boom 9 in advance. The reference image Is is an image that serves as a reference for determining the tip portion of the boom 9. The reference image Is is composed of at least a portion of an image of the tip portion of the boom 9 in an arbitrarily determined reference posture. In other words, the reference image Is is an image that shows the reference posture of the tip portion of the boom 9. In this embodiment, the reference posture is the boom 9 at a predetermined hoisting angle. In other words, the reference image Is is composed of at least a portion of an image taken from above of the tip portion of the boom 9 at a predetermined hoisting angle.

The judgment image detection unit 26b is part of the image processing device 26, and detects an image of a portion of the current image P that corresponds to the reference image Is as a judgment image Ij. The judgment image Ij is an image of the tip portion of the boom 9 in the current image P relative to the reference attitude of the tip portion of the boom 9. In other words, the judgment image Ij is an image for calculating the relative attitude of the tip portion of the boom 9 in the current image P relative to the reference image Is. The judgment image Ij is composed of an image of at least a portion of the tip portion of the boom 9 in the current image P.

The judgment image detection unit 26b can calculate feature points of the selected reference image Is. Furthermore, the judgment image detection unit 26b can calculate feature points from the current image P. The judgment image detection unit 26b can compare the feature points of the reference image Is with the feature points of the current image P, and detect the part of the current image P that corresponds to the reference image Is as the judgment image Ij.

The attitude determination unit 26c is part of the image processing device 26, and determines the orientation D of the current extension direction of the boom 9 relative to the reference orientation of the luggage W. The attitude determination unit 26c is configured to be able to calculate the shape, direction, and inclination of the judgment image Ij relative to the reference image Is by comparing the reference image Is with the judgment image Ij. The attitude determination unit 26c can calculate the tip coordinates B of the boom 9 in the coordinate system of the current image P relative to the reference orientation of the tip of the boom 9 and the orientation D of the extension direction of the boom 9 in the coordinate system of the current image P from the coordinates of each part of the reference image Is and the coordinates of each part of the judgment image Ij.

The image processing device 26 can calculate the rotation center coordinate C, which is the coordinate of the rotation center of the boom 9 in the coordinate system of the image P, from the posture information of the boom 9, such as the size of the detected tip portion of the boom 9 in the image P, the extension direction of the boom 9, the boom hoisting angle of the boom 9, the extension length of the boom 9, the rotation angle of the boom 9, and the shape and dimensions of the boom 9. The image processing device 26 superimposes the calculated tip coordinate B, rotation center coordinate C, and direction D of the boom 9 on the image P displayed on the display device 24.

The image processing device 26 can calculate a transformation formula for transforming the coordinate system of the image P into a crane coordinate system with the boom 9's center of rotation as the origin, based on the tip coordinate B of the boom 9 in the coordinate system of the image P, the boom 9's center of rotation coordinate C, and the boom 9's posture information. The image processing device 26 can transform the tip coordinate B and center of rotation coordinate C in the coordinate system of the image P into the transformed tip coordinate Bc and center of rotation coordinate Cc in the crane coordinate system using the transformation formula.

The image processing device 26 also acquires the position of the pass point at the tip of the boom 9 input from the display device 24 as pass point coordinates T1 (x1, y1), T2 (x2, y2), ... (hereinafter simply referred to as "pass point coordinate T(n)", where n is an integer indicating the order) which are coordinates in the coordinate system of the image P. Furthermore, the image processing device 26 converts the pass point coordinate T(n) into transformed pass point coordinates Tc1 (xc1, yc1), Tc2 (xc2, yc2), ... (hereinafter simply referred to as "transformed pass point coordinate Tc(n)", where n is an integer indicating the order) which are coordinates in the crane coordinate system using a conversion formula (see FIG. 5). The image processing device 26 transmits the transformed pass point coordinate Tc(n) to the control device 27.

As shown in FIG. 2, the control device 27 controls each actuator of the crane 1. The control device 27 is provided in the cabin 17. The control device 27 may be configured with a CPU, ROM, RAM, HDD, etc. connected via a bus, or may be configured with a one-chip LSI, etc. The control device 27 stores various programs and data for controlling the operation of the actuators, switching valves, sensors, etc., and for processing image data. The control device 27 includes an image processing device 26.

As shown in Figures 2 and 4, the control device 27 is connected to the traveling operation tool 18 and the crane operation tool 19. The control device 27 can acquire the operation signals of the traveling operation tool 18 and the crane operation tool 19. The control device 27 generates a control signal for the vehicle 2 based on the operation signal generated by the operation of the traveling operation tool 18. Similarly, the control device 27 generates a control signal for the crane device 6 based on the operation signal generated by the operation of the crane operation tool 19. The control device 27 can transmit the generated control signal to each actuator of the crane device 6.

The control device 27 can calculate the vertical height (vertical coordinate) from the ground of the main hook 10a suspended from the main wire rope 14 or the sub-hook 11a suspended from the sub-wire rope 16 from the attitude information of the boom 9 and the amount of the main wire rope 14 paid out by the main winch 13 or the amount of the sub-wire rope 16 paid out by the sub-winch 15. In addition, when controlling the boom 9 so that the tip of the boom 9 passes through a passing point, the control device 27 can control the main winch 13 or the sub-winch 15 so that the vertical coordinate from the ground of the main hook 10a or the sub-hook 11a does not fluctuate. In other words, when the crane 1 transports the cargo W by automatic transport control, the height of the cargo W from the ground is kept constant.

The control device 27 is connected to the image processing device 26 by wire or wirelessly. The control device 27 can transmit attitude information of the boom 9 to the image processing device 26. The control device 27 can acquire multiple transformed passing point coordinates Tc(n) which are the coordinates of the passing points of the tip of the boom 9 in the crane coordinate system from the image processing device 26. The control device 27 can acquire transformed tip coordinates Bc which are the coordinates of the tip of the boom 9 in the crane coordinate system and transformed rotation center coordinates Cc which are the coordinates of the rotation center of the boom 9 from the image processing device 26.

The control device 27 can generate a control signal for the tip of the boom 9 to pass through the multiple transformation pass-through point coordinates Tc(n) from the acquired transformation tip coordinates Bc and multiple transformation pass-through point coordinates Tc(n). The control device 27 can transmit a control signal for passing through the generated transformation pass-through point coordinates Tc(n) to each actuator of the crane device 6. In other words, the control device 27 can perform automatic transport control to automatically transport the luggage W along the pass-through points at a predetermined speed.

The control device 27 can acquire an operation signal of the main winch 13 or the sub winch 15 from the image processing device 26. The control device 27 generates a control signal of the main winch 13 or the sub winch 15 based on the operation signal of the main winch 13 or the sub winch 15. The control device 27 can transmit the generated control signal to each actuator of the crane device 6. The control device 27 can manually change the vertical position of the luggage W during automatic transport control.

The crane 1 configured in this manner can move the vehicle 2 to any position by operating the travel operating device 18. The crane 1 can also transport cargo W to any position by rotating, raising, lowering, and extending the boom 9 by operating the crane device operating device 19. The crane 1 can also hoist or lower cargo W with the main winch 13 or the like by operating the crane device operating device 19. Meanwhile, the crane 1 can perform automatic transport control to control the tip of the boom 9 to pass through a passing point set based on the image P captured by the imaging device 20.

Image processing by the image processing device 26 will be described below with reference to Figures 4 and 5. Note that the image P captured by the image capture device 20 includes the crane 1 and the work site around the crane 1. The crane 1 is assumed to be in a state in which the cargo W is hoisted to a predetermined height by the sub-winch 15. The unmanned aerial vehicle 21 of the image capture device 20 is assumed to be hovering above the tip of the boom 9 before the start of transportation (see Figure 3). Furthermore, even if the boom 9 moves, the unmanned aerial vehicle 21 is assumed to maintain the hovering position it had before the boom 9 moved. At this time, the image processing device 26 is assumed to perform sway correction so that the position on the screen of a target object such as a structure that has been automatically detected or designated by the operator does not fluctuate.

The image processing device 26 acquires an image P from above the crane 1 taken by the camera 22 via the communication device 25 every unit time (see FIG. 4). In addition, the image processing device 26 acquires posture information of the boom 9 from the control device 27 every unit time. The image processing device 26 displays the acquired image P on the display device 24.

The image processing device 26 detects a judgment image Ij, which is an image of the tip of the boom 9, from the latest image P captured by the camera 22. The image processing device 26 prestores a reference image Is, which is an image of the tip of the boom 9 detected using a convolutional neural network (CNN) from images of the boom tips of various cranes 1 (see FIG. 4).

The image processing device 26 detects a judgment image Ij, which is an image of the tip of the boom 9, from the latest image P by image classification using CNN based on the reference image Is. From the detected judgment image Ij and the attitude information of the boom 9, the image processing device 26 calculates the direction D of the axis of the boom 9 in the judgment image Ij, the tip coordinate B indicating the position of the tip of the boom 9 on the axis in the coordinate system of the image P, and the rotation center coordinate C indicating the rotation center of the boom 9 in the coordinate system of the image P.

Specifically, the image processing device 26 calculates the length from the tip to the base of the boom 9 in the coordinate system of the image P taken from above, based on the boom 9 attitude information, which is the width of the boom 9 tip, the boom 9 elevation angle, and the boom 9 extension length. The image processing device 26 calculates the rotation center coordinate C from the calculated tip coordinate B, axis direction D, and the length from the tip to the base of the boom 9. As a result, the image processing device 26 can convert the coordinate system of the image P from the tip coordinate B, rotation center coordinate C, axis direction D, and boom 9 attitude information into a crane coordinate system with the boom 9 rotation center as the origin and the forward direction of the vehicle 2 of the crane 1 as the Y-axis direction. The image processing device 26 converts the tip coordinate B and rotation center coordinate C in the coordinate system of the image P into the converted tip coordinate Bc and converted rotation center coordinate Cc in the crane coordinate system. Note that, although the crane coordinate system is an orthogonal coordinate in this embodiment, it may be a polar coordinate.

The operator inputs the passing points on image P where the tip of boom 9 will pass. The operator inputs the passing points as a line that is a collection of multiple points. That is, the operator inputs the passing points of the tip of boom 9 as the trajectory line of the tip of boom 9 by, for example, tracing the image P displayed on display device 24 with his/her finger. The operator sets the passing points while checking the status of the boom 9 of crane 1, which is displayed in image P, as well as the lifting position of the cargo W and the installation position of the cargo W, from above the crane 1. This allows the operator to intuitively set a transport route that avoids obstacles based on the layout of structures at the work site.

The image processing device 26 calculates passing point coordinates T(1), T(2), ... passing point coordinate T(n), which are the coordinates of passing points at predetermined intervals on the trajectory line input by the operator on the screen of the display device 24. Next, the image processing device 26 converts them into transformed passing point coordinates Tc(n) in the coordinate system of the image P. In other words, the image processing device 26 converts the passing point coordinates T(n) indicating the position on the image P into transformed passing point coordinates Tc(n) indicating the actual position at the work site (see Figure 4). The image processing device 26 transmits the converted transformed passing point coordinates Tc(n) to the control device 27.

The control device 27 calculates the amount of operation of each actuator for moving the tip of the boom 9 to each converted passing point coordinate Tc(n) based on the converted rotation center coordinate Cc and the converted tip coordinate Bc. The control device 27 also calculates the amount of operation of the sub winch 15 for maintaining the vertical coordinate of the suspended load W.

When the control device 27 acquires the operation signal of the automatic transport control switch 24a displayed on the display device 24 via the image processing device 26, it controls each actuator based on the calculated operation amount of each actuator. As a result, the tip of the boom 9 of the crane 1 moves along the converted passing point coordinate Tc(n) of the passing point (trajectory line) input by the operator (see the two-dot chain line diagram in FIG. 5). At this time, the control device 27 reels in and reels in the sub wire rope 16 to maintain the vertical height of the sub hook 11a from the ground. When the control device 27 acquires the operation signal of the winch operation switch 24b that operates the sub winch 15, it generates a control signal for the sub winch 15 according to the operation signal. The control device 27 releases the control that maintains the vertical coordinate of the cargo W and changes the vertical coordinate of the cargo W based on the operation signal of the operation switch of the sub winch 15.

The steps of the automatic transport control of the crane 1 are described below with reference to FIG. 6. In this embodiment, the photographing device 20 is an unmanned aerial vehicle 21 equipped with a camera 22 that photographs the crane 1 below while hovering at a predetermined position. The control device 27 and image processing device 26 are also assumed to have attitude information of the boom 9.

As shown in FIG. 6, in step S110 of the automatic transport control of the crane 1, the image processing device 26 included in the control device 27 acquires the latest image P of the work site including the crane 1 captured by the camera 22. The image processing device 26 transitions to step S120.

In step S120, the image processing device 26 detects a judgment image Ij, which is an image of the tip of the boom 9, based on the reference image Is from the most recently acquired image P by image classification using CNN. The image processing device 26 transitions to step S130.

In step S130, the image processing device 26 calculates the direction D of the axis of the boom 9 in the judgment image Ij from the detected judgment image Ij and the attitude information of the boom 9, the tip coordinate B indicating the position of the tip of the boom 9 on the axis in the coordinate system of the image P, and the rotation center coordinate C indicating the rotation center of the boom 9 in the coordinate system of the image P. The image processing device 26 transitions to step S140.

In step S140, the image processing device 26 calculates a conversion equation for converting the coordinate system of the image P into the crane coordinate system from the attitude information of the boom 9, the tip coordinate B, and the rotation center coordinate C. The image processing device 26 transitions to step S150.

In step S150, the image processing device 26 calculates the position of the passing point on the screen input from the display device 24 as passing point coordinates T(n) indicated in the coordinate system of the image P. The image processing device 26 then proceeds to step S160.

In step S160, the image processing device 26 converts the turning center coordinate C, the tip coordinate B, and the pass point coordinate T(n) into the converted turning center coordinate Cc, the converted tip coordinate Bc, and the converted pass point coordinate Tc(n) using the calculated conversion formula. The image processing device 26 transitions to step S170.

In step S170, when the control device 27 receives an operation signal from the automatic transport control switch 24a, it operates each actuator so that the tip of the boom 9 passes through the conversion passing point coordinate Tc(n) at a predetermined speed. The control device 27 transitions to step S180.

In step S180, the control device 27 determines whether or not an operation signal of the winch operation switch 24b for operating the sub winch 15 has been acquired.
As a result, when an operation signal of the winch operation switch 24b for operating the sub winch 15 is acquired, the control device 27 shifts the step to step S190.
On the other hand, if the control device 27 has not received an operation signal from the winch operation switch 24b for operating the sub winch 15, the control device 27 shifts the step to step S200.

In step S190, the control device 27 operates the sub winch 15 based on the operation signal of the winch operation switch 24b. The control device 27 transitions to step S200.

In step S200, the control device 27 determines whether or not the tip of the boom 9 has passed through all the transformed passing-point coordinates Tc(n).
As a result, when the tip of the boom 9 has passed through all the transformed passing-through point coordinates Tc(n), that is, when the luggage W has been transported to the destination position, the control device 27 ends the step.
On the other hand, if the tip of the boom 9 has not passed through all of the transformed passing-through point coordinates Tc(n), the control device 27 transitions to step S180.

With this configuration, the image processing device 26 automatically calculates the coordinates of the rotation center of the crane 1 and the coordinates of the tip of the boom 9 by, for example, comparing the image P including the crane 1 and the surroundings of the crane 1 captured by the camera 22 with the image P of the tip of the boom 9 previously acquired. The control device 27 can convert the coordinate system of the image P into a crane coordinate system based on the rotation center of the crane 1 by calculating the rotation center coordinate C and tip coordinate B of the boom 9 using the image processing device 26.

The image processing device 26 can easily calculate the converted passing point coordinate Tc(n) of the passing point coordinate T(n) specified on the image P based on the rotation center coordinate C. Therefore, the operator of the crane 1 can set the passing point of the tip of the boom 9 on the image P while checking the situation around the crane 1 on the image P in real time. In addition, the control device 27 performs automatic transport control to control each actuator so that the tip of the boom 9 passes the passing point at a predetermined speed. At this time, the crane 1 maintains the vertical coordinate of the luggage W suspended from the hook even if the boom 9 is raised and lowered so that the tip of the boom 9 passes the passing point specified by the operator. Therefore, the operator does not need to control the vertical position of the luggage W in accordance with the movement of the boom 9. As a result, the operator can transport the luggage W to the target position with easy operations while grasping the situation of the work site in real time using the crane 1.

Next, a crane 1A according to a second embodiment of the present invention will be described with reference to Figures 7 and 8. Note that the crane 1A according to the following embodiments will refer to the same crane 1 shown in Figures 1 to 11, and the names, drawing numbers, and symbols used in the description will be used to refer to the same thing. Therefore, in the following embodiments, specific descriptions of the same points as in the embodiments already described will be omitted, and the description will focus on the differences.

As shown in FIG. 7, the crane 1A is a mobile crane that can be moved to any location. The crane 1A has a vehicle 2, a crane device 6, an imaging device 20, a display device 24, a first GNSS receiver 28, a direction sensor 29, a second GNSS receiver 30, a communication device 25, an image processing device 26, and a control device 27.

The first GNSS receiver 28 is a receiver that constitutes the Global Navigation Satellite System. The first GNSS receiver 28 receives ranging radio waves from satellites and calculates the absolute coordinates of the receiver, which are latitude, longitude, and altitude. The first GNSS receiver 28 is provided at the rotation center of the boom 9. The crane 1A can obtain the absolute coordinates of the rotation center of the boom 9 by using the first GNSS receiver 28.

The orientation sensor 29 detects the orientation. The orientation sensor 29 is composed of a geomagnetic sensor such as an MR sensor or an MI sensor, a GNSS receiver having two antennas, etc. The orientation sensor 29 detects the orientation in the extension direction of the boom 9. The orientation sensor 29 is provided at the center of rotation of the boom 9. The orientation sensor 29 may also be configured to be shared by the first GNSS receiver 28.

The second GNSS receiver 30 is a receiver that constitutes the Global Navigation Satellite System. The second GNSS receiver 30 receives ranging radio waves from satellites and calculates the receiver's absolute coordinates, which are latitude, longitude, and altitude. The second GNSS receiver 30 is provided on the unmanned aerial vehicle 21 of the imaging device 20. The second GNSS receiver 30 uses the calculated absolute coordinates to enable the crane 1A to obtain the absolute coordinates of the camera 22 provided on the unmanned aerial vehicle 21 via the second GNSS receiver 30.

The communication device 25 can receive absolute coordinates calculated by the first GNSS receiver 28. The communication device 25 can receive absolute coordinates calculated by the second GNSS receiver 30.

As shown in FIG. 8, the image processing device 26 processes the image P captured by the camera 22. The image processing device 26 can calculate a conversion formula for converting the coordinate system of the image P into an absolute coordinate system from the absolute rotation center coordinate Ca, which is the absolute coordinate of the rotation center received by the first GNSS receiver 28, the direction D of the extension direction of the boom 9 calculated by the direction sensor 29, the absolute camera coordinate Ia, which is the absolute coordinate of the camera 22 received by the second GNSS receiver 30, and the attitude information of the boom 9. The image processing device 26 can calculate the absolute tip coordinate Ba, which is the absolute coordinate of the tip of the boom 9, using the conversion formula. In addition, the image processing device 26 can calculate the absolute pass point coordinate Ta(n) from the pass point coordinate T(n) of the pass point of the tip of the boom 9 input to the display device 24 using the conversion formula. The image processing device 26 transmits the calculated absolute pass point coordinate Ta(n) to the control device 27.

Image processing by the image processing device 26 will be described below with reference to FIG. 9. Note that the image P captured by the imaging device 20 includes the crane 1A and the work site around the crane 1A. The crane 1A is assumed to be in a state in which the cargo W is hoisted to a predetermined height by the sub-winch 15. The unmanned aerial vehicle 21 of the imaging device 20 is assumed to be hovering above the tip of the boom 9 before the start of transportation. The unmanned aerial vehicle 21 is assumed to maintain the hovering position even if the boom 9 moves. At this time, the image processing device 26 is assumed to perform sway correction so that the position on the screen of the target object, such as a building designated automatically or by the pilot, does not fluctuate.

The image processing device 26 of the crane 1A acquires an image P from above the crane 1A taken by the camera 22 via the communication device 25 every unit time. In addition, the image processing device 26 acquires posture information of the boom 9 from the control device 27 every unit time. The image processing device 26 displays the acquired image P on the display device 24.

The image processing device 26 acquires the absolute turning center coordinate Ca from the first GNSS receiver 28, acquires the direction D of the extension direction of the boom 9 from the direction sensor 29, and acquires the absolute camera coordinate Ia from the second GNSS receiver 30. Based on the positional relationship between the absolute camera coordinate Ia and the absolute turning center coordinate Ca, the image processing device 26 calculates the absolute tip coordinate Ba from the absolute turning center coordinate Ca, the direction D of the extension direction of the boom 9, and the attitude information of the boom 9. Furthermore, based on the absolute camera coordinate Ia and the angle of view information of the camera 22, the image processing device 26 calculates a conversion formula for converting coordinates in the coordinate system of the image P into absolute coordinates.

The image processing device 26 calculates passing point coordinates T(n), which are the coordinates of multiple passing points input by the operator on the screen of the display device 24. The image processing device 26 calculates absolute passing point coordinates Ta(n) from the calculated passing point coordinates T(n) using a relational expression calculated based on the absolute camera coordinates Ia and the angle of view information of the camera 22. In other words, the image processing device 26 calculates absolute passing point coordinates Ta(n), which indicate the actual position at the work site in absolute coordinates, from the passing point coordinates T(n), which indicate the position on the image P. The image processing device 26 transmits the calculated absolute passing point coordinates Ta(n) to the control device 27.

The control device 27 calculates, for each absolute pass point coordinate Ta(n), the amount of operation of each actuator for moving the tip of the boom 9 to each absolute pass point coordinate Ta(n) from the acquired multiple absolute pass point coordinates Ta(n) and absolute tip coordinate Ba. The control device 27 also calculates the amount of operation of the sub winch 15 for maintaining the vertical coordinate of the suspended cargo W.

The automatic transport control of the crane 1A will be described below with reference to FIG. 10. In this embodiment, the photographing device 20 is an unmanned aerial vehicle 21 equipped with a camera 22 that photographs the crane 1A below while hovering at a predetermined position. The control device 27 and image processing device 26 are also assumed to have attitude information of the boom 9.

As shown in FIG. 10, in step S110 of the automatic transport control of the crane 1A, the image processing device 26 included in the control device 27 acquires the latest image P captured by the camera 22. The image processing device 26 transitions to step S220.

In step S220, the image processing device 26 acquires the absolute turning center coordinate Ca from the first GNSS receiver 28, acquires the orientation D in the extension direction from the orientation sensor 29, and acquires the absolute camera coordinate Ia from the second GNSS receiver 30. The image processing device 26 transitions to step S230.

In step S230, the image processing device 26 calculates the absolute tip coordinate Ba from the absolute turning center coordinate Ca, the direction D of the extension direction of the boom 9, and the attitude information of the boom 9 based on the positional relationship between the absolute camera coordinate Ia and the absolute turning center coordinate Ca. The image processing device 26 transitions to step S240.

In step S240, the image processing device 26 calculates a conversion equation for converting the coordinates in the coordinate system of the image P into absolute coordinates based on the absolute camera coordinates Ia and the angle of view information of the camera 22. The image processing device 26 transitions to step S140.

In step S150, the image processing device 26 calculates the position of the passing point input from the display device 24 as passing point coordinates T(n) indicated in the coordinate system of the image P. The image processing device 26 then proceeds to step S260.

In step S260, the image processing device 26 calculates the absolute pass-point coordinate Ta(n) from the pass-point coordinate T(n) using a conversion formula that converts coordinates in the coordinate system of the image P into absolute coordinates. The image processing device 26 transitions to step S170.

In step S300, the control device 27 determines whether or not the tip of the boom 9 has passed through all the absolute pass-point coordinates Ta(n).
As a result, when the tip of the boom 9 has passed through all the absolute passing point coordinates Ta(n), that is, when the luggage W has been transported to the destination position, the control device 27 ends the step.
On the other hand, if the tip of the boom 9 has not passed through all of the absolute pass-through point coordinates Ta(n), the control device 27 transitions to step S180.

With this configuration, the control device 27 calculates the absolute tip coordinate Ba from the absolute rotation center coordinate Ca of the boom 9, the orientation D of the extension direction of the boom 9, the absolute camera coordinate Ia, the angle of view information of the camera 22, and the attitude information of the boom 9. The control device 27 also calculates a conversion formula for converting coordinates in the coordinate system of the image P into absolute coordinates based on the absolute camera coordinate Ia and the angle of view information of the camera 22. This makes it possible to convert the coordinate system of the image P into an absolute coordinate system.

The operator of the crane 1A can therefore set a passing point for the tip of the boom 9 on the image P while checking the situation around the crane 1A on the image P in real time. The control device 27 also performs automatic transport control to control each actuator so that the tip of the boom 9 passes through the passing point. This allows the operator to transport the cargo W to the destination position with simple operations while grasping the situation at the work site in real time using the crane 1A.

In each of the above-described embodiments, the crane 1, 1A may further include an obstacle photographing device 31 that monitors contact between the luggage W and an obstacle such as a building.
As shown in FIG. 11 , as another embodiment of the second embodiment of the present invention, a crane 1A is equipped with an obstacle photography device 31 including an obstacle unmanned aerial vehicle 32 and an obstacle camera 33 .

The obstacle unmanned aerial vehicle 32 is an autonomous unmanned aerial vehicle (drone) that can be remotely controlled and fly autonomously by control signals. The obstacle unmanned aerial vehicle 32 is, for example, a multicopter with multiple propellers. The obstacle unmanned aerial vehicle 32 is configured to be able to send and receive control signals and the like to and from the remote control terminal 23. As a result, the obstacle unmanned aerial vehicle 32 is configured to be able to move to any position by the remote control terminal 23.

The obstacle camera 33 is provided on the obstacle unmanned aerial vehicle 32. The obstacle camera 33 is configured to be able to capture images in any direction from the obstacle unmanned aerial vehicle 32. The obstacle camera 33 is configured to be able to acquire control signals and the like from the remote control terminal 23. The obstacle camera 33 is also configured to be able to transmit captured image data to the image processing device 26. This allows the obstacle camera 33 to be operated by the remote control terminal 23 to start and stop capture, determine the capture direction, zoom, and perform other operations.

When automatic transport control is started, the control device 27 controls the obstacle unmanned aerial vehicle 32 to follow the luggage W while maintaining a position where it can photograph the luggage W and its surroundings. When the control device 27 acquires an operation signal from an operation switch that operates the sub-winch 15, it generates a control signal for the sub-winch 15 in response to the operation signal. The control device 27 releases the control that maintains the hanging position of the luggage W and changes the hanging position of the luggage W based on the operation signal from the operation switch of the sub-winch 15. At the same time, the control device 27 automatically controls the obstacle unmanned aerial vehicle 32 to follow the hanging position of the luggage W. The control device 27 acquires an image P of the luggage W and its surroundings by the obstacle camera 33.

When changing the vertical position of the luggage W during automatic transport control, the operator of the crane 1A equipped with the obstacle photography device 31 can adjust the vertical position of the luggage W while checking the image P from the obstacle photography device 31. In addition, the operator of the crane 1A equipped with the obstacle photography device 31 can move the obstacle unmanned aerial vehicle 32 to any position from the remote control terminal 23. This allows the operator to transport the luggage W to the desired position with simple operations while grasping the situation at the work site in real time using the crane 1A.

In each of the above-described embodiments, the control device 27 of the crane 1, 1A stops the boom 9 when the tip of the boom 9 passes all passing points during automatic transport control. However, the control device 27 may be configured to stop the movement of the boom 9 when it receives a stop signal from the operator.

The above-mentioned embodiment is merely a representative example, and various modifications can be made without departing from the gist of the embodiment. Of course, the present invention can be implemented in various other forms, and the scope of the present invention is indicated by the claims, and further includes the equivalent meanings described in the claims, and all modifications within the scope of the claims.

REFERENCE SIGNS LIST 1 Mobile crane 9 Boom 20 Shooting device 24 Display device 21 Unmanned aerial vehicle 22 Camera 26 Image processing device 27 Control device P Image C Rotation center coordinate B Tip coordinate T(n) Passing point coordinate

Claims (7)

A mobile crane having a swivel base and a freely raising and lowering boom,
an imaging device that images the mobile crane equipped with the boom and the surroundings of the mobile crane from above;
a display device that displays the image captured by the imaging device and is capable of inputting a movement trajectory of the boom;
A control device for controlling an actuator of the mobile crane,
The control device includes:
A mobile crane which acquires an image including the mobile crane and the surrounding area captured by the imaging device, displays the image on the display device, calculates the coordinates of the boom's center of rotation in the coordinate system of the image and the coordinates of the tip of the boom in the coordinate system of the image from the image and posture information of the boom, calculates the coordinates in the coordinate system of the image of a passing point through which the tip of the boom passes, which is set on the image, and controls an actuator of the mobile crane so that the tip of the boom passes through the passing point at a predetermined speed. The control device includes:
2. The mobile crane according to claim 1, wherein an image of the tip of the boom is detected from an image captured by the photographing device, and a coordinate of a center of rotation of the boom in the image and a coordinate of the tip of the boom in a coordinate system of the image are calculated from a direction and a size of the image of the tip of the boom and attitude information of the boom. A mobile crane having a swivel base and a freely raising and lowering boom,
an imaging device that images the mobile crane equipped with the boom and the surroundings of the mobile crane from above;
A first GNSS receiver for detecting absolute coordinates of a rotation center of the boom;
An orientation sensor for detecting an orientation of the extension direction of the boom;
A second GNSS receiver for detecting absolute coordinates of the photographing device;
a display device that displays the image captured by the imaging device and is capable of inputting a movement trajectory of the boom ;
A control device for controlling an actuator of the mobile crane,
The control device includes:
acquiring an image including the mobile crane and the surroundings of the mobile crane photographed by the photographing device, displaying the image on the display device, acquiring absolute coordinates of a rotation center of the boom from the first GNSS receiver, acquiring an orientation of the extension direction of the boom from the orientation sensor, acquiring absolute coordinates of the photographing device from the second GNSS receiver, and calculating absolute coordinates of the tip of the boom from the absolute coordinates of the rotation center of the boom, the absolute coordinates of the photographing device, and attitude information of the boom,
a mobile crane which calculates absolute coordinates of a passing point through which the tip of the boom, which is set at a position on the image, from the absolute coordinates of the boom's rotation center, the absolute coordinates of the camera, the absolute coordinates of the tip of the boom, and angle of view information of the camera, and controls an actuator of the mobile crane so that the tip of the boom passes through the absolute coordinates of the passing point at a predetermined speed. The imaging device is
A mobile crane as described in any one of claims 1 to 3, comprising an unmanned aerial vehicle and a camera mounted on the unmanned aerial vehicle. The control device includes:
5. A mobile crane as claimed in any one of claims 1 to 4, wherein the vertical coordinate from the ground of a hook suspended from a wire rope configured to be freely reeled in and out from the boom by a winch is calculated, and when the boom is controlled so that the tip of the boom passes through the passing point, the winch is controlled so as to maintain the vertical coordinate of the hook from the ground. The control device includes:
The mobile crane according to claim 5, wherein when an operation signal of a winch operating tool for operating the winch is acquired, the winch is controlled based on the operation signal. The method further includes an obstacle photographing device including an unmanned aerial vehicle and a camera provided on the unmanned aerial vehicle,
The obstacle photographing device is
The mobile crane according to claim 5 or 6, wherein an image of the periphery of the hook is photographed while following the hook.

Images