JP6919817B2 - Crane control system and control method - Google Patents

Description

The present invention relates to a crane control system and a control method, and more particularly to a crane control system and a control method for improving cargo handling efficiency.

As a device to remoteize the operation of the crane using the hook, the projection corresponding point corresponding to the projection point (expected landing center point) on the horizontal plane of a specific height such as the ground surface of the hook is added to the image taken by the camera. A display device has been proposed (see, for example, Patent Document 1).

This device superimposes the projection corresponding points on the image taken by the camera, so that the operator who operates the crane from a remote location can see the position of the projection point on the horizontal plane at a specific height such as the ground surface of the hook. It makes it easier to recognize.

Japanese Unexamined Patent Publication No. 2016-179889

By the way, an image taken by a camera has a problem that a delay occurs between the time when the image is taken and the time when the image is actually displayed on the display device. Therefore, in the device described in Patent Document 1, a delayed image is displayed to the operator in a remote operation in which the operator cannot actually confirm the operation of the crane and time the operation. This is a factor that causes the timing of crane operation to shift. Such an operation timing shift due to an image delay lowers the alignment accuracy and lowers the cargo handling efficiency.

An object of the present invention is to provide a crane control system and a control method capable of eliminating delays in operation timing and improving cargo handling efficiency.

The control device of the crane of the present invention that achieves the above object is provided by a girder portion extending in one direction, a trolley supported by the girder portion and moving in the extending direction of the girder portion, and a wire from the trolley. In a crane control system including a suspended hanger, a camera installed on the trolley and sequentially capturing an image including a landing target surface on which the hanger and the hanger below it land. The display device that sequentially displays the images captured by the camera, the position acquisition device that acquires the plane positions of the hanger and the landing target surface, the camera, the display device, and the position acquisition device. With the connected control device, the hanger and the landing target surface are aligned in a plan view by the control device based on the plane position acquired by the position acquisition device. An upper layer image is created in which a display that can identify the fact is drawn on a transparent background, and the created upper layer image and the image captured by the camera are laminated and displayed on the display device asynchronously with each other, and the upper layer image is displayed. The image is transparent from the transparent background of the above.

The method for controlling a crane of the present invention that achieves the above object is to lower a hanger suspended by a wire from a trolley that is supported by a girder portion extending in one direction and travels in the extending direction of the girder portion. In a method for controlling a crane in which the lower end of the hanger or the lower end of the suspended load grasped by the hanger is landed on the landing target surface, the hanger and the hanger below the hanger are measured by a camera installed in the trolley. At the same time as sequentially capturing images including the landing target surface on which the implant is landed, the position acquisition device acquires the plane positions of the hanger and the landing target surface, and the control device acquires the image captured by the camera. And the upper layer image created by drawing a display on the transparent background that can identify that the hanger and the landing target surface are aligned in a plan view based on the acquired plane position. Are laminated on the display device asynchronously with each other, and the image is transmitted from the transparent background of the upper layer image.

The present invention is a display device for asynchronously displaying an upper layer image on which a display that can identify that the hanger and the landing target surface are aligned in a plan view is drawn on a transparent background and an image captured by a camera. To display in layers. Therefore, according to the present invention, while confirming the operation of the hanging tool by the image captured by the camera, even if there is a delay before the image is displayed on the display device, the identifiable display is in a state close to real time. Can be displayed on the display device with. As a result, by having the operator operate the hanger according to the identifiable display, it is advantageous to eliminate the timing difference of the operation, and when the hanger is landed on the landing target surface, the hanger Can be aligned with the landing target surface.

That is, according to the present invention, it is possible to accurately teach the operator the timing of performing the operation of lowering the hanging tool by the identifiable display without any time delay. By landing the hanger according to this identifiable indication, even an inexperienced person can benefit from shortening the time required for aligning the hanger with the landing target surface, and can improve cargo handling efficiency.

It is a block diagram which illustrates the embodiment of the control system of the crane of this invention. It is a block exemplifying the control system of FIG. It is explanatory drawing which illustrates the lower layer image of FIG. It is explanatory drawing which illustrates the middle layer image of FIG. It is explanatory drawing which illustrates the positional relationship with the spreader of FIG. 1 and a landing target surface. It is explanatory drawing which illustrates the upper layer image of FIG. It is explanatory drawing which illustrates the laminated image of FIG. 2, and is the state which the spreader and the landing target surface are not aligned with each other in a plan view. It is explanatory drawing which illustrates the laminated image of FIG. 2, and is the state which the spreader and the landing target surface are aligned with each other in a plan view. It is a relational figure which illustrates the relationship between the elapsed time in the control system of FIG. 2, the display cycle of the lower layer image and the middle layer image, and the change of the state quantity. FIG. 5 is a relationship diagram illustrating the relationship between the elapsed time in the control system of FIG. 2, the display cycle of the upper layer image, and the change in the state quantity. It is the first half part of the flow chart which illustrates embodiment of the crane control method of this invention. It is the latter half part of the flow chart which illustrates embodiment of the crane control method of this invention. 11 is a relationship diagram illustrating the relationship between each step of the flow charts of FIGS. 11 and 12 and the elapsed time.

Hereinafter, embodiments of the crane control system and control method of the present invention will be described. In the figure, the extending direction of the girders (11, 12) of the crane 10 is shown in the x1 direction, the direction in which the crane 10 is moved by the traveling device 16 orthogonal to the x1 direction is shown in the y1 direction, and the vertical direction is shown in the z1 direction. .. Further, in the laminated image 30 described later, the vertical direction is shown in the x2 direction and the horizontal direction is shown in the y2 direction.

The control system 20 of the crane 10 illustrated in FIGS. 1 and 2 is a system in which an operator existing in a facility separated from the crane 10 causes the crane 10 to handle the cargo of the container 31 by remote control.

As illustrated in FIG. 1, the crane 10 is a crane that handles cargo of a container 31 for a ship anchored at a quay. The crane 10 is suspended by a boom 11 and a girder 12 as girders extending in the x1 direction, a trolley 13 supported by the boom 11 and the girder 12 and moving in the x1 direction, and a trolley 13 as a hanging tool by a wire 14. The spreader 15 and the spreader 15 are provided.

The spreader 15 is suspended by a plurality of wires 14 suspended downward from the central portion of the trolley 13 in a plan view, and the wires 14 are wound or unwound to be moved up and down.

The girders (11, 12) are supported on the upper part of the leg structure (17, 18), the boom 11 projects from the leg structure to the sea side in the x1 direction, and the girder 12 extends from the leg structure to the land side in the x1 direction. Overhang. In the leg structure, a traveling device 16 capable of traveling along a rail laid on the quay in the y1 direction is installed at the lower end, and a plurality of leg bodies 17 extending upward from the traveling device 16 and a plurality of legs 17 It is composed of a horizontal beam 18 connecting the legs 17.

The crane 10 is provided with a traversing device (not shown) for traversing the trolley 13 and an elevating device (not shown) for raising and lowering the spreader 15 by winding or unwinding the wire 14 in the machine room 19. The crane 10 may be a crane in which the traversing device is installed in the trolley 13 or a crane in which the elevating device is installed in the trolley 13.

The control system 20 includes a camera 21, a display device 22, position acquisition devices 23 and 24, a control device 25, a crane control device 26, an operation device 27, and a notification device 28. The control system 20 is a system in which the control device 25 displays the stacked image 30 on the display device 22 based on the information acquired by the camera 21 and the position acquisition devices 23 and 24. The laminated image 30 is an image in which the lower layer image 32 captured by the camera 21 and the upper layer image 33 created based on the information acquired by the position acquisition devices 23 and 24 are asynchronously superimposed and displayed on the display device 22. ..

The camera 21 is a camera installed in the trolley 13 that sequentially captures an unprocessed image 35, which is an image including a spreader 15 and a landing target surface 34 below the spreader 15, at predetermined imaging cycles T1. In the present specification, the lower part of the spreader 15 also includes the area directly below the spreader 15 on which the spreader 15 lands in the z1 direction. Further, the landing target surface 34 is a surface on which the lower end of the spreader 15 or the lower end of the container 31 gripped by the spreader 15 is landed, and the ground, the ground contact surface of the ship, or the upper surface of the container 31 is exemplified. Further, in the present specification, the unprocessed image 35 is an image that has not been subjected to any image processing after being captured by the camera 21. The image processing includes image correction for correcting parameters (pixels, density, amount of information, etc.) related to the appearance of the image, and editing of the image itself (selecting a part of the image, cropping, enlarging / reducing, etc.). It shall be distinguished from image editing.

Specifically, the camera 21 is installed at the land-side end of the trolley 13 in the x1 direction in a side view, and the camera lens 21a is directed obliquely downward. The optical axis A1 of the camera 21 is tilted with respect to the z1 direction, with the upper side facing the land side in the x1 direction and the lower side facing the sea side in the x1 direction in a side view. The optical axis A1 is a virtual axis that passes through the camera center point C1 that is the center of the camera lens 21a and extends in the direction perpendicular to the surface of the camera lens 21a. The optical axis A1 is tilted counterclockwise by an angle θ1 with respect to the z1 direction. The angle θ1 is set to an angle at which the landing target surface 34 (upper surface of the container 31) existing directly below the spreader 15 and the spreader 15 in the z1 direction is reflected on the unprocessed image 35 captured by the camera 21.

In this way, the control system 20 directs the camera lens 21a diagonally downward and overlooks the spreader 15 and the container 31 loaded on the ship, so that the operation of the control system 20 is installed in the trolley in a crane whose operation is not remoted. It becomes possible to reproduce the view of the operator who boarded the room. As a result, the operator existing in the facility away from the crane 10 can confirm the operation of the spreader 15 by the image captured by the camera 21, and the remote control becomes possible while the addressee is in the driver's cab. It is advantageous to improve the operability of the operation.

It is sufficient that the camera 21 can sequentially capture the unprocessed image 35 including the spreader 15 and the landing target surface 34. The camera 21 may be appropriately changed in the installation location and the direction of the camera lens 21a except that the camera 21 is installed directly above the spreader 15 in the z1 direction and the camera lens 21a is directed directly below in the z1 direction. The imaging cycle T1 is not particularly limited, but at the latest, a cycle equal to or less than the sampling cycle T2 of the position acquisition devices 23 and 24 described later is exemplified.

The display device 22 is composed of a single display that sequentially displays the stacked images 30. The display device 22 is installed in a facility separated from the crane 10 and sequentially displays the laminated image 30 to an operator who remotely controls the crane 10. It is sufficient for one display device 22 to display the laminated image 30 corresponding to one crane 10, and one display device 22 can switchably display a plurality of laminated images 30 corresponding to the plurality of cranes 10. You may. In this way, by configuring the display device 22 with one display for the operator, the operator can pay attention to the information aggregated by the laminated image 30 displayed on one display and perform remote control. Become. Therefore, according to the control system 20, the frequency of movement of the operator's line of sight is reduced as compared with the configuration of displaying on a plurality of displays, the operator's attention is distracted, and information is overlooked. It is advantageous to avoid this, and the safety for remote control can be improved.

The position acquisition devices 23 and 24 are installed on the trolley 13 and are devices that add a height position to the plane positions of the spreader 15 and the landing target surface 34 to sequentially acquire three-dimensional positions at predetermined sampling periods T2. .. Specifically, the position acquisition devices 23 and 24 are devices that sequentially acquire the three-dimensional positions of the spreader center point C2, the landing target surface center point C3, and the expected landing center point C4 for each sampling period T2. The sampling period T2 is not particularly limited.

The position acquisition device 23 is composed of a shape recognition system using a two-dimensional laser sensor or a three-dimensional laser sensor, and the position acquisition device 24 is composed of a runout angle sensor. The position acquisition devices 23 and 24 are not limited to the above configuration as long as they can acquire at least the plane positions of the spreader 15 and the landing target surface 34. For example, as a device for acquiring the plane position of the spreader 15, a GPS receiver and a device for measuring the feeding amount and the winding amount of the wire 14 are exemplified. Further, as a device for acquiring the plane position of the landing target surface 34, a laser sensor installed on the spreader 15 is exemplified. Further, the position acquisition device 23 may be two devices, a device for acquiring the plane position of the spreader 15 and a device for acquiring the plane position of the landing target surface 34.

The spreader 15 has a substantially rectangular shape with the short side facing the x1 direction and the long side facing the y1 direction in a plan view, and the spreader center point C2, which is the center of the spreader 15, is an intersection of rectangular diagonal lines. .. The landing target surface 34 has a substantially rectangular shape in a plan view similar to the spreader 15 or the container 31, and the landing target surface center point C3, which is the center of the landing target surface 34, is an intersection of rectangular diagonal lines. It becomes. The expected landing center point C4 is a point where the spreader center point C2 is vertically projected onto the horizontal plane 36 including the landing target surface 34. The horizontal plane 36 is a plane that extends horizontally including the landing target plane 34, and is a plane that is perpendicular to the z1 direction.

The control device 25 is hardware composed of a CPU that performs various information processing, an internal storage device that can read and write programs and information processing results used for performing the various information processing, and various interfaces. The control device 25 is installed in a facility separated from the crane 10 or on the ground in the vicinity of the crane 10, and is electrically connected to the display device 22 and the operation device 27 via a signal line (not shown). Further, the control device 25 is communicably connected to the camera 21, the position acquisition devices 23, and 24 via a communication line such as a wireless antenna or an optical fiber.

In the control device 25, the arrangement information of the containers 31 loaded on the ship is input in advance as the situation around the landing target surface 34. The situation around the landing target surface 34 is appropriately updated as the cargo handling progresses by the crane 10. In addition to the ship, the arrangement information of the container 31 in the container yard where the container 31 is stored may be input.

Like the control device 25, the crane control device 26 is hardware composed of a CPU, an internal storage device, various interfaces, and the like. The crane control device 26 is installed in the machine room 19 of the crane 10 and is electrically connected to the traversing device of the trolley 13, the lifting device of the spreader 15, and the traveling device 16 via a signal line (not shown). Further, the crane control device 26 is communicably connected to the operation device 27 via a wireless antenna. The crane control device 26 may be connected to the operation device 27 via a wired communication such as an optical cable instead of the wireless antenna.

The notification device 28 is a device that emits sound or light toward the operator, and the status of the spreader 15 by emitting sound or light to the operator operating the crane 10 by the operating device 27 by looking at the display device 22. It is a device that can instantly notify. Examples of the notification device 28 include a speaker, a buzzer, a rotating light, and a blinking light. Examples of the situation of the spreader 15 include a situation in which the spreader 15 and the landing target surface 34 are close to each other in a relative positional relationship, and a situation in which the spreader 15 may come into contact with other than the landing target surface 34 and a warning needs to be issued. NS.

As illustrated in FIG. 2, the control device 25 has a first image creation unit 40, a second image creation unit 41, a display control unit 42, and a notification control unit 43 as each functional element, and creates a second image. The unit 41 includes a calculation unit 44, a specific display creation unit 45, a relative position display creation unit 46, and a warning display creation unit 47. Each functional element in this embodiment is stored as a program in the internal storage device, and is executed by the CPU in a timely manner. In addition to the program, electric circuits in which each function independently functions are also exemplified as each functional element. Further, each functional element may be composed of a PLC (Programmable Logical Controller), and the control device 25 may be an aggregate of a plurality of PLCs.

The first image creating unit 40 is a functional element that performs image processing on the unprocessed image 35 captured by the camera 21 to create a lower layer image 32 and an enlarged image 37 (middle layer image 38). Specifically, the first image creation unit 40 creates a lower layer image 32 in which only the image correction of the image processing is performed on the unprocessed image 35, and then the created lower layer image 32 is subjected to image processing. This is a functional element that edits the image and creates a middle layer image 38 composed of a plurality of enlarged images 37.

As illustrated in FIG. 3, the lower layer image 32 is an image obtained by performing image correction on the unprocessed image 35 captured by the camera 21 to make it clear, and the image size is equivalent to the image size of the unprocessed image 35. It is an image arranged in the lowermost layer of the laminated image 30. The lower layer image 32 is an image showing the landing target surface 34 set on the upper surface of the spreader 15 and the container 31 to be handled. The lower layer image 32 is displayed on the display device 22 later than the image taken by the camera 21 by the delay time T3 required for the image correction. The delay time T3 includes the propagation delay time until the unprocessed image 35 captured by the camera 21 is transmitted as data and received by the control device 25, and the display device 22 after the image is corrected by the control device 25. It shall include the propagation delay time until it is displayed.

As illustrated in FIG. 4, the middle layer image 38 is an image in which a plurality of enlarged images 37 are arranged by subjecting the lower layer image 32 to image editing and enlarging a part of a portion reflected in the lower layer image 32. The middle layer image 38 is an image whose image size is the same as that of the unprocessed image 35 and is laminated on the layer above the lower layer image 32 in the laminated image 30.

The enlarged image 37 is an enlarged image of each of the four corners of the landing target surface 34 reflected in the lower layer image 32, and the image size is smaller than the image size of the unprocessed image 35 and is arranged at the four corners of the transparent background 38a. In the present specification, the transparent background 38a (the same applies to the transparent background 33a described later) is a background in which an image arranged in a lower layer is transparently displayed, and for example, a transparent background is exemplified. Further, as the transparent background 38a, a background (blue background or green background) in chroma key composition in which a specific color component is made transparent and another image is synthesized on the transparent portion is also exemplified. The middle layer image 38 is displayed later by the delay time T4 required for image editing of the enlarged image 37 after the lower layer image 32 is displayed on the display device 22. It is assumed that the delay time T4 also includes the propagation delay time like the delay time T3.

As illustrated in FIG. 2, the second image creation unit 41 creates figures, scales, and numerical values based on the measured values acquired by the position acquisition devices 23 and 24, and creates an upper layer image 33 in which they are appropriately arranged. It is a functional element to be used. Specifically, the second image creation unit 41 indicates that the spreader 15 and the landing target surface 34 are in a state of being aligned in a plan view based on the measured values acquired by the position acquisition devices 23 and 24. Figures, scales, and numerical values are drawn as identifiable displays, displays relating to the relative positional relationship between the spreader 15 and the landing target surface 34, and warning displays for avoiding contact between the spreader 15 and other than the landing target surface 34. This is a functional element for creating the upper layer image 33.

The calculation unit 44 calculates the calculated value required for the identifiable display, the calculated value required for the display related to the relative positional relationship, and the calculated value required for determining the warning display based on the measured values acquired by the position acquisition devices 23 and 24. It is a functional element. Specifically, the calculation unit 44 determines the landing target surface center point C3 and the expected landing center point C4 based on the three-dimensional positions acquired by the position acquisition devices 23 and 24 as the calculated values required for the identifiable display. It is a functional element for calculating the two-dimensional coordinates in the laminated image 30. Further, the calculation unit 44 determines the height in the z1 direction between the spreader 15 and the landing target surface 34 based on the three-dimensional positions acquired by the position acquisition devices 23 and 24 as the calculated values required for displaying the relative positional relationship. It is a functional element that calculates the distance L5 in the H7 and x1 directions, and the skew angle θ5, which is the rotation angle around the vertical axis of the spreader 15. In addition, the calculation unit 44 determines the distance (L5 + L6) from the center of the landing target surface 34 to one end of the spreader 15 in plan view and the distance (L5 + L6) from the center of the landing target surface 34 as the calculated values required for determining the warning display. It is a functional element that calculates the distance (L7 + L8) to one end of the container 31 existing in the surroundings.

As illustrated in FIG. 5, the calculation unit 44 includes a landing target angle θ3, which is an angle formed by a line segment A3 connecting the optical axis A1 and the camera center point C1 to the landing target surface center point C3, and an optical axis. The expected landing angle θ4, which is the angle formed by the line segment A4 connecting A1 and the camera center point C1 to the expected landing center point C4, is calculated. Then, the calculation unit 44 uses the calculated landing target formed angle θ3 and the predicted landing target surface center point C3 and the predicted landing center point C4 in the laminated image 30 (upper layer image 33). The dimensional coordinates (Lθ3, Lθ4) are calculated. Further, the calculation unit 44 calculates the height H7, the distance L5, the skew angle θ5, the distance (L5 + L6), and the distance (L7 + L8) in the process of calculating the two-dimensional coordinates.

In the following, it is assumed that the y1 coordinates of the plane positions (x1 coordinates, y1 coordinates) of the spreader center point C2, the landing target surface center point C3, and the expected landing center point C4 are substantially the same. In the crane 10, when the spreader 15 is moved up and down in the z1 direction, the movement in the y1 direction by the traveling device 16 is completed, and the alignment in the y1 direction is completed. Further, the shaking of the spreader 15 when the spreader 15 is moved up and down in the z1 direction mainly occurs in the moving direction of the trolley 13.

When the calculation unit 44 calculates the landing target angle θ3 and the expected landing angle θ4, the center point C5 in the x1 direction is simple when the spreader 15 and the wire 14 are regarded as a simple pendulum. It shows the position that serves as the fulcrum of the pendulum. Further, the reference surface 39 which is the reference of the height in the z1 direction is the lower end surface of the trolley 13, and is regarded as a horizontal plane perpendicular to the vertical line passing through the center point C5 which is the fulcrum thereof. The center point C5 and the reference surface 39 can be arbitrarily set. The center point C5 is not necessarily the center of the trolley 13 in the x1 direction, and the reference surface 39 may serve as a reference for the height in the z1 direction. It is not always the lower end surface of 13.

In the figure, H1 and H2 are measured values acquired by the position acquisition device 23, H3 to H5 are preset set values, and H6 and H7 are calculated by the calculation unit 44 based on the measured values H1 and H2. The heights are calculated so that the lower part in the z1 direction is positive and the upper part is negative. The measured value H1 is the height from the lower end of the position acquisition device 23 to the landing target surface 34. The measured value H2 is the height from the lower end of the position acquisition device 23 to the upper surface of the spreader 15. The set value H3 is the height from the horizontal plane 36 to the lower end of the position acquisition device 23. The set value H4 is the height from the reference surface 39 to the camera center point C1. The set value H5 is the height from the upper end to the lower end of the spreader 15. The calculated value H6 is the height from the camera center point C1 to the landing target surface 34. The calculated height H7 is the height in the z1 direction between the lower end of the spreader 15 and the landing target surface 34, and is the height until the spreader 15 lands on the landing target surface 34. The set value H5 is the height from the upper end of the spreader 15 to the lower end of the container 31 when the spreader 15 is gripping the container 31. Further, the measured value H2 may be the height (H2 + H3) from the reference surface 39 to the upper surface of the spreader 15 when acquired based on the rope feeding amount.

In the figure, L1 is a measured value acquired by the position acquisition device 23, L2, L3, L6, L7, and L8 are preset set values, and L4 and L5 are calculated values calculated by the calculation unit 44. , The right side in the x1 direction is positive, and the left side is negative. The measured value L1 is the length from the center of the position acquisition device 23 in the x1 direction to the center point C3 of the landing target surface. The set value L2 is the length from the center point C5 to the center of the position acquisition device 23 in the x1 direction. The set value L3 is the length from the center point C5 to the camera center point C1. The set value L6 is the length from the center to the side surface of the spreader 15 in the x1 direction. The set value L7 is half the length from the center to the side surface of the container 31 in the x1 direction, that is, the length of the container 31 in the x1 direction. The set value L8 is the distance between the containers 31 adjacent to each other in the x1 direction. The calculated value L4 is the length from the camera center point C1 to the landing target surface center point C3. The calculated distance L5 is the length (distance) in the x1 direction between the landing target surface center point C3 and the expected landing center point C4, and is the amount of deviation of the spreader 15 with respect to the landing target surface 34. .. As the set values L7 and L8, values based on the array information of the container 31 input in advance to the control device 25 may be used. However, since the stacking position of the container 31 is different for each ship, the set value L8 is preferably a value calculated based on the stacking shape acquired by the position acquisition device 23 configured by the shape recognition system.

In the figure, θ1 is a preset set value, θ2 and θ5 are measured values acquired by the position acquisition device 24, and θ3 and θ4 are calculated values calculated by the calculation unit 44, respectively, with respect to the z1 direction. The slope is positive for counterclockwise and negative for clockwise. The set value θ1 is the inclination of the optical axis A1 of the camera 21. The measured value θ2 is the runout angle of the spreader 15 in the x1 direction, and the intersection of the vertical line and the vertical line passing through the lower end surface of the position acquisition device 24 composed of the runout angle sensor and the center point C5 and the spreader center point C2 It is the angle formed by the connecting line segment. The skew angle θ5, which is a measured value, is a rotation angle of the spreader 15 around the vertical axis. The landing target formed angle θ3 is an angle formed by the line segment A3 connecting the landing target surface center point C3 and the camera center point C1 and the optical axis A1. The expected landing angle θ4 is the angle formed by the line segment A4 connecting the predicted landing center point C4 and the camera center point C1 and the optical axis A1.

The calculation unit 44 calculates the two-dimensional coordinates of the landing target surface center point C3 in the laminated image 30 based on the angle θ3 formed by the landing target. Specifically, the calculation unit 44 calculates the landing target formed angle θ3 using the following mathematical formulas (1) to (3), and based on the calculated landing target formed angle θ3 and the number of pixels of the camera 21. , The x2 coordinates (Lθ3: see FIG. 6) in the laminated image 30 (or the upper layer image 33) of the landing target surface center point C3 are calculated.

The calculation unit 44 calculates the two-dimensional coordinates of the predicted landing center point C4 in the laminated image 30 based on the predicted landing angle θ4. Specifically, the calculation unit 44 calculates the expected landing angle θ4 using the above formulas (2) and (3) and the following formulas (4) and (5), and the calculated suspended angle θ4 and the camera. Based on the number of pixels of 21, the x2 coordinates (Lθ4: see FIG. 6) in the laminated image 30 of the expected implantation center point C4 are calculated.

算出部４４は、下記の数式（６）を用いて、着床までの距離として高さＨ７を算出する。

The calculation unit 44 calculates the height H7 as the distance to the landing by using the following mathematical formula (6).

The specific display creation unit 45 is a landing target indicating the landing target surface center point C3 as a display that can be specified based on the two-dimensional coordinates ((Lθ3, 0), (Lθ4, 0)) calculated by the calculation unit 44. It is a functional element that draws the expected landing mark 33c indicating the mark 33b and the expected landing center point C4 on the transparent background 33a of the upper layer image 33.

The relative position display creation unit 46 sets the height H7 in the z1 direction between the spreader 15 and the landing target surface 34 calculated by the calculation unit 44, the distance L5 in the x1 direction, and the skew angle θ5 around the vertical axis of the spreader 15. Based on this, it is a functional element that draws the height display 33d, the distance display 33e, and the angle display 33f on the transparent background 33a of the upper layer image 33 as displays relating to the relative positional relationship.

The warning display creating unit 47 determines whether or not to issue a warning display for avoiding contact between the spreader 15 and the landing target surface 34, based on the distances (L5 + L6) and (L7 + L8) calculated by the calculation unit 44. When it is determined that the warning display is to be issued, the warning display 33g is a functional element in the transparent background 33a of the upper layer image 33. It is preferable that the warning display creating unit 47 determines whether or not to issue a warning display based on the added distance of the skew runout of the spreader 15.

As illustrated in FIG. 6, the upper layer image 33 visualizes the relationship between the spreader 15 and the landing target surface 34 by figures, numerical values, and scales based on the measured values acquired by the position acquisition devices 23 and 24. It is an image in which figures, numerical values, and scales are drawn by each of the specific display creation unit 45, the relative position display creation unit 46, and the warning display creation unit 47. The upper layer image 33 is an image in which the image size is the same as that of the unprocessed image 35 and is laminated on the upper layer than the lower layer image 32 in the laminated image 30. Specifically, in the upper layer image 33, the landing target mark 33b indicating the landing target surface center point C3 and the expected landing mark 33c indicating the expected landing center point C4 are drawn as identifiable displays. Further, the upper layer image 33 is a height display 33d, x1 indicating the height (distance until the spreader 15 lands) H7 in the z1 direction between the spreader 15 and the landing target surface 34 as a display relating to the relative positional relationship. A distance display 33e indicating the distance L5 in the direction and an angle display 33f indicating the skew angle θ5 around the vertical axis of the spreader 15 are drawn. In addition, the upper layer image 33 is drawn with a warning display 33g that encourages avoidance of contact between the spreader 15 and the landing target surface 34. The upper layer image 33 is displayed later on the display device 22 after the position acquisition devices 23 and 24 acquire the three-dimensional position by the delay time T5 due to the slight calculation delay. The delay time T5 also includes the propagation delay time like the delay time T3.

As illustrated in FIG. 2, the display control unit 42 asynchronously transfers the lower layer image 32 and the middle layer image 38 created by the first image creation unit 40 and the upper layer image 33 created by the second image creation unit 41. It is a functional element that creates a laminated image 30 by laminating and synthesizing, and displays the created laminated image 30 on the display device 22.

As used herein, asynchronous means that the images are not synchronized with each other. For example, when the images are taken by the camera 21 and the measured values are acquired by the position acquisition devices 23 and 24 at the same time, the time required to create the upper layer image 33 is longer than the time required to create the lower layer image 32 and the middle layer image 38. Since the length is shorter, the display device 22 displays the upper layer image 33, which is completed first. That is, as soon as each image is created, it is laminated and combined as a laminated image 30 and displayed on the display device 22.

As illustrated in FIGS. 7 and 8, in the laminated image 30, the upper layer image 33, the middle layer image 38, and the lower layer image 32 are laminated in this order from the upper layer to the lower layer, and the transparent background 33a and the middle layer image 38 of the upper layer image 33 are laminated. This is an image in which the lower layer image 32 is transmitted from the transparent background 38a of the above. The laminated image 30 is an image in which the time axes of the upper layer image 33, the middle layer image 38, and the lower layer image 32 are different from each other. Specifically, the laminated image 30 has a figure drawn on the upper layer image 33, a numerical value, a scale time, an image time drawn on the lower layer image 32, and an image time drawn on the enlarged image 37 of the middle layer image 38. There is a delay from the actual time in the order of, and the time of the figure, the numerical value, and the scale drawn on the upper layer image 33 becomes the closest to the actual time.

In the laminated image 30, the landing target mark 33b and the expected landing mark 33c, which are identifiable displays, are arranged in the central portion of the display area (screen) of the display device 22, and other displays (33d to 33f) relating to the relative positional relationship. ) And the warning display 33g are arranged at the outer edge of the display area. The enlarged image 37 is also arranged at the outer edge of the display area. In this way, information that can specifically determine the operation timing of the spreader 15 is arranged in the central portion of the display area, and information that assists in determining the operation timing is arranged in the outer edge portion of the display area. By doing so, it is possible to give superiority or inferiority to the information. This is advantageous for conspicuous the most important identifiable display in determining the operation timing of the spreader 15, and the frequency of overlooking the operation timing can be reduced. It is desirable that the spreader 15 and the landing target surface 34 in the lower layer image 32 are transparently displayed in the central portion of the display area of the display device 22 in the same manner as the identifiable display.

As illustrated in FIGS. 9 and 10, the change in the state quantity of each image with the elapsed time and the display cycle of each image are compared. The state quantity referred to here is, for example, the amount of movement of the spreader 15. The display cycle of the lower layer image 32 and the display cycle of the middle layer image 38 are the same as the imaging cycle T1 of the camera 21, that is, the display of the unprocessed image 35, and the display cycle of the upper layer image 33 is the sampling of the position acquisition devices 23 and 24. It is the same cycle as the cycle T2. That is, while the actual state quantity changes continuously, the state quantity in each image changes with each display cycle of each image. In other words, the state quantity in each image does not change until the display cycle of each image elapses. On the other hand, the display cycle of the lower layer image 32 has a delay time T3 with respect to the imaging cycle T1, and the display cycle of the middle layer image 38 has a time obtained by summing the delay time T3 and the delay time T4 with respect to the imaging cycle T1. In the display cycle of the image 33, a delay time T5 occurs with respect to the sampling cycle T2. As a result, the lower layer image 32 at the same time t4 becomes an image at time t2, the middle layer image 38 becomes an image at time t1, and the upper layer image 33 becomes an image at time t3. The times α, β, and γ in the figure indicate the time elapsed since the state quantity changed in each image.

The notification control unit 43 notifies information regarding the relative positional relationship between the spreader 15 and the landing target surface 34 based on the calculated value transmitted from the calculation unit 44 of the second image creation unit 41 to the relative position display creation unit 46. It is a functional element to notify 28. Further, when the notification control unit 43 determines that the warning display creation unit 47 issues a warning display based on the determination in the warning display creation unit 47 of the second image creation unit 41, the spreader 15 and the landing target surface 34 It is also a functional element that notifies the notification device 28 of a warning for avoiding contact with other than. For example, the notification device 28 notifies a situation in which the spreader 15 and the landing target surface 34 are close to each other by increasing the sound over time or narrowing the interval at which the sound is emitted. Further, the notification device 28 instantly raises the warning sound to warn that the spreader 15 may come into contact with a surface other than the landing target surface 34. In addition, when the spreader 15 lands on the landing target surface 34, the notification device 28 makes a sound imitating the landing sound generated when the spreader 15 lands, and the spreader 15 makes a landing target surface 34. Notify that you have landed on.

As illustrated in FIGS. 11 to 13, the control method of the crane 10 using the control system 20 is performed when the operator remotely controls the spreader 15 to land on the designated landing target surface 34. The method. Specifically, in this control method, the upper layer image 33, the middle layer image 38, and the lower layer image 32 are laminated and displayed on the display device 22 asynchronously with each other, and the transparent backgrounds 33a and 38a of the upper layer image 33 and the middle layer image 38 are displayed. This is a method of transmitting the lower layer image 32. 11 and 12 are flow charts exemplifying the flow of data, and FIG. 13 exemplifies the timing of each image generation process displayed at the elapsed time t4.

This control method is a method in which the operator starts when the operation of landing the spreader 15 by the operation device 27 is performed, and is periodically repeated until the operation is completed. The double line in the figure indicates parallel processing. In the present specification, the parallel processing is a processing performed in parallel with each other by different devices and functional elements, and includes a case where the processing times are not synchronized with each other.

When the operator performs an operation of landing the spreader 15 by the operating device 27, the camera 21 sequentially captures the unprocessed image 35 including the spreader 15 and the landing target surface 34 in each imaging cycle T1 (S110). In parallel with this step S110, the position acquisition devices 23 and 24 sequentially acquire the measured values (H1, H2, L1, θ2, θ5) for each sampling cycle T2 (S120).

When the camera 21 captures the unprocessed image 35, the first image creation unit 40 of the control device 25 performs the image correction of the image processing on the unprocessed image 35 to create the lower layer image 32 (S210). .. Next, the first image creation unit 40 performs image editing in the image processing on the lower layer image 32 to create the middle layer image 38 (S220).

When the position acquisition devices 23 and 24 acquire the measured values, the calculation unit 44 of the second image creation unit 41 determines the landing target angle θ3, the expected landing angle θ4, and the height based on the measured value and the set value. H7, distance L5, skew angle θ5, distance (L5 + L6), and distance (L7 + L8) are calculated (S310).

Next, the specific display creation unit 45 creates an upper layer image of the landing target surface center point C3 and the predicted landing center point C4 based on the landing target angle θ3, the predicted landing angle θ4, and the number of pixels of the camera 21. The two-dimensional coordinates in 33 are calculated, and a landing target mark 33b indicating the landing target surface center point C3 and a predicted landing mark 33c indicating the expected landing center point C4 are created (S320). In parallel with this step, the relative position display creating unit 46 creates the height display 33d, the distance display 33e, and the angle display 33f based on the height H7, the distance L5, and the skew angle θ5 (S330). Further, in parallel with these steps, the warning display creation unit 47 determines whether or not there is a possibility of contact between the spreader 15 and the landing target surface 34 (S340). In this step, when the distance (L5 + L6) calculated by the calculation unit 44 becomes greater than or equal to the distance (L7 + L8), it is determined that there is a possibility of contact between the spreader 15 and the landing target surface 34 other than the distance (L7 + L8). When L5 + L6) is less than the distance (L7 + L8), it is determined that there is no possibility of contact between the spreader 15 and the landing target surface 34. Next, when it is determined that there is a possibility of contact (S340: YES), the warning display creating unit 47 creates a warning display 33g (S350). On the other hand, when it is determined that there is no possibility of contact (S340: NO), the warning display creating unit 47 erases the warning display 33g (S360). In this way, each display is created, and the creation of the upper layer image 33 is completed.

Next, the display control unit 42 stacks the upper layer image 33, the middle layer image 38, and the lower layer image 32 to create the stacked image 30 (S410). At this time, the display control unit 42 creates the laminated image 30 by sequentially laminating the images sent from the respective creating units at the present time without synchronizing the images. Next, when the display control unit 42 displays the created laminated image 30 on the display device 22 (S420), the process returns to the start.

In parallel with the steps in the display control unit 42, the notification control unit 43 notifies the height H7, the distance L5, and the skew angle θ5 by the notification device 28, or the spreader 15 and the landing target surface 34 other than the spreader 15 and the landing target surface 34. There is a possibility of contact, a warning for avoiding the contact is notified by the notification device 28 (S510), and the process returns to the start.

As illustrated in FIG. 7, in the laminated image 30, the landing target mark 33b and the expected landing mark 33c are separated from each other, and the spreader 15 and the landing target surface 34 are not aligned in a plan view. Is. In the laminated image 30, the two-dimensional coordinates of the landing target mark 33b are (Lθ3, 0) with respect to the origin (0, 0) indicating the optical axis A1 as the center thereof, and the two-dimensional coordinates of the expected landing mark 33c are It becomes (Lθ4, 0). At this time, the expected landing angle θ4 is larger than the landing target angle θ3, and the expected landing mark 33c is displayed on the sea side in the x2 direction from the landing target mark 33b.

Further, in the laminated image 30, the height display 33d displays the height H7 as the distance until the spreader 15 lands on the landing target surface 34, and it can be seen that it takes some time to land. Further, the distance display 33e and the angle display 33f indicate that the relative positional relationship between the spreader 15 and the landing target surface 34 is deviated. In addition, in the laminated image 30, the warning display 33g displays a warning indicating the possibility that the spreader 15 may come into contact with a surface other than the landing target surface 34 in a graphic indicating the direction in which the spreader 15 is moved.

The operator sees the laminated image 30 and determines that it is not the timing to land the spreader 15, and performs the operation of aligning the spreader 15. Specifically, the operator moves the trolley 13 to the land side in the x2 direction while confirming the movement of the spreader 15 by the lower layer image 32 of the laminated image 30, or the timing at which the spreader 15 and the landing target surface 34 are aligned. Aim.

As illustrated in FIG. 8, in the laminated image 30, the landing target mark 33b and the expected landing mark 33c overlap each other, and the spreader 15 and the landing target surface 34 are aligned in a plan view. be. In the laminated image 30, the two-dimensional coordinates of the landing target mark 33b are (Lθ3, 0) with respect to the origin (0, 0) indicating the optical axis A1 as the center thereof, and the two-dimensional coordinates of the expected landing mark 33c are also It becomes (Lθ3,0). At this time, the expected landing angle θ4 is equal to the landing target angle θ3.

Further, in the laminated image 30, the height display 33d displays the height H7 as the distance until the spreader 15 lands on the landing target surface 34, and it can be seen that the height H7 is immediately before landing. Further, the distance display 33e and the angle display 33f indicate that the relative positional relationship between the spreader 15 and the landing target surface 34 is not deviated. In addition, in the laminated image 30, since the warning display 33g is not displayed, it is displayed that there is no possibility that the spreader 15 comes into contact with anything other than the landing target surface 34.

The operator sees the laminated image 30 and determines that it is time to land the spreader 15, and performs an operation of landing the spreader 15 on the landing target surface 34. Specifically, the operator lowers the spreader 15 toward the landing target surface 34 while confirming the movement of the spreader 15 by the lower layer image 32 of the laminated image 30. Further, while the operator lowers the spreader 15 toward the landing target surface 34, the operator directs the spreader 15 toward the landing target surface 34 while confirming the alignment of the spreader 15 with the enlarged image 37 of the laminated image 30. And take it down.

According to the above control system 20, by displaying the lower layer image 32 and the upper layer image 33 in an asynchronous manner, the lower layer image 32 captures the operation of the spreader 15 while confirming the operation of the spreader 15 by the lower layer image 32 captured by the camera 21. Even if there is a delay before the image is displayed on the display device 22, the upper layer image 33 created based on the measured values acquired by the position acquisition devices 23 and 24 can be displayed on the display device 22 in a state close to real time. .. As a result, by having the operator operate the hanging tool according to the identifiable display (33b, 33c) drawn on the upper layer image 33, it becomes advantageous to eliminate the deviation of the operation timing, and the spreader 15 is worn. When the floor target surface 34 is landed, the spreader 15 and the landing target surface 34 can be aligned with each other.

That is, according to the control system 20, it is possible to accurately teach the operator the timing of performing the operation of lowering the spreader 15 by the identifiable display (33b, 33c) without any time delay. By landing the spreader 15 according to this identifiable display, even an inexperienced person can benefit from shortening the time required for aligning the spreader 15 with the landing target surface 34, and can improve cargo handling efficiency.

For example, when the operator operates the spreader 15 based only on the lower layer image 32 captured by the camera 21, even if the optimum timing for lowering the spreader 15 on the landing target surface 34 in the lower layer image 32 is determined, the actual timing is determined. At this time, the spreader 15 has reached another position. That is, in such a case, the operator must drive while predicting the future several seconds ahead from the situation of the spreader 15 that can be grasped from the lower layer image 32.

On the other hand, according to the control system 20, the display device 22 displays the lower layer image 32 captured by the camera 21 and the upper layer image 33 created based on the measured values without synchronizing them. The status of the spreader 15 can be displayed with a delay time T5 shorter than the delay time T3 of the lower layer image 32. This eliminates the need to drive while predicting the future situation of the spreader 15, and allows an inexperienced operator whose prediction accuracy is lower than that of a skilled operator to accurately perform the alignment operation of the spreader 15. can.

Further, since the control system 20 may cause a delay in the lower layer image 32 captured by the camera 21, the control system 20 is compared with a method of reducing the display delay time of the image by using a device capable of displaying the image in real time. It is possible to reduce the introduction cost.

In addition, the control system 20 provides the operator with the spreader 15 by the upper layer image 33 even when the lower layer image 32 captured by the camera 21 is unclear due to the influence of the surrounding environment (weather, cargo handling work time (nighttime, etc.)) of the crane 10. It becomes possible to accurately convey the situation. As a result, the influence of the surrounding environment of the crane 10 can be eliminated, and the crane 10 can be loaded and unloaded by remote control.

In addition to the overlapping condition of the landing target mark 33b and the expected landing mark 33c, characters can be specified as a display that can identify that the spreader 15 and the landing target surface 34 are in the aligned state in a plan view. Or symbols may be displayed at the moment when the spreader 15 and the landing target surface 34 are aligned in a plan view. Further, the notification device 28 may emit a sound or light that can identify that the spreader 15 and the landing target surface 34 are in the aligned state in a plan view.

The control system 20 determines the degree of overlap of the landing target mark 33b and the expected landing mark 33c over time from a state in which the spreader 15 and the landing target surface 34 are not aligned in a plan view to a state in which they are aligned. It is displayed by the laminated image 30 as a change. As a result, the timing for lowering the spreader 15 can be determined from the overlapping condition of the landing target mark 33b and the expected landing mark 33c, which change with time. Therefore, the plane surface between the spreader 15 and the landing target surface 34 can be determined. This is advantageous in reducing the time required for visual alignment.

In particular, the control system 20 performs an operation of lowering the spreader 15 when the spreader 15 and the landing target surface 34 are aligned in a plan view even when the spreader 15 swings periodically like a pendulum. Will be possible. This is advantageous in reducing the waiting time until the runout of the spreader 15 is settled.

The runout cycle (amplitude) of the spreader 15 may be measured, and the predicted implantation center point C4 may be used as the predicted point predicted from that cycle. When the crane 10 is remotely controlled, there may be a time delay until each device is driven with respect to the operation of the operating device 27. Due to this time delay, when the spreader 15 actually lands on the landing target surface 34 even if the timing for lowering the spreader 15 is planned based on the overlapping condition of the landing target mark 33b and the expected landing mark 33c. The position may not match. Therefore, by using a predicted point in consideration of the operation time delay as the predicted landing center point C4, it is advantageous to eliminate the positional deviation due to various time delays.

In addition to the landing target mark 33b and the expected landing mark 33c, the control system 20 may draw a spreader mark indicating the spreader center point C2, which is the center of the spreader 15, on the upper layer image 33. The two-dimensional coordinates in the upper image 33 of the spreader center point C2 can be calculated based on the angle formed with respect to the optical axis A1 as in the case of the landing target surface center point C3 and the expected landing center point C4. In addition, a mark instead of the spreader mark may be provided directly on the spreader 15. In this way, by displaying the spreader center point C2 in addition to the landing target mark 33b and the expected landing mark 33c, the number of marks increases and the positional relationship between the spreader 15 and the landing target surface 34 can be grasped. Will be advantageous to.

The control system 20 is a system for remotely controlling the crane 10, and reproduces the view of the operator in the driver's cab with the lower layer image 32 and the middle layer image 38 captured by the camera 21. Therefore, the camera lens 21a of the camera 21 is directed diagonally downward, and it is advanced to find a state in which the camera lens 21a is aligned with the landing target surface 34 several tens of meters below the spreader 15 only by those images. Requires skill. Therefore, the control system 20 creates an upper layer image 33 on which the landing target mark 33b and the expected landing mark 33c indicating the landing target surface center point C3 and the expected landing center point C4 are drawn. This is advantageous for grasping the positional relationship between the spreader 15 and the landing target surface 34, and even an inexperienced person can perform positioning with the same high accuracy as a skilled person.

As described above, the camera lens 21a of the camera 21 is directed obliquely downward, and in order to calculate the two-dimensional coordinates of the stacked image 30, it is necessary to grasp the positional relationship of the camera 21 with respect to the optical axis A1. Therefore, the control system 20 calculates the two-dimensional coordinates of the landing target surface center point C3 and the expected landing center point C4 in the laminated image 30 based on the landing angle θ3 and the expected landing angle θ4. .. This is advantageous for calculating the two-dimensional coordinates of the landing target surface center point C3 and the expected landing center point C4 in the laminated image 30 with high accuracy.

The control system 20 measures the measured values H1, H2, L1, and θ2 for each sampling cycle T2 by the position acquisition devices 23 and 24, and the landing target surface center point C3 and the expected landing center point C4 in the upper layer image 33. Calculate the dimensional coordinates. Therefore, even if the position of the landing target surface 34 or the position of the spreader 15 changes due to the shaking of the ship due to the waves, the two-dimensional coordinates in the upper layer image 33 can be calculated according to the changes. This is advantageous for highly accurate alignment.

The control system 20 allows the operator to grasp the current situation of the spreader 15 in more detail by drawing the display regarding the relative positional relationship between the spreader 15 and the landing target surface 34 as figures, numerical values, and scales on the upper layer image 33. Can be done. For example, even when the landing target mark 33b and the expected landing mark 33c match in the upper layer image 33, the height H7 until the spreader 15 lands on the landing target surface 34 may be high. Further, due to the skew runout generated in the spreader 15, the position of the spreader 15 may shift about the vertical axis with respect to the landing target surface 34 in a plan view. In such a case, even if the spreader 15 is lowered by the operation of the operator, the spreader 15 may shift with respect to the landing target surface 34 in a plan view when landing. Therefore, by indicating the current position and posture of the spreader 15 to the operator by displaying the relative positional relationship between the spreader 15 and the landing target surface 34, it is possible to achieve highly accurate operation timing. For example, it becomes possible to teach the timing of slowing down the speed at which the spreader 15 is lowered, which is advantageous for reducing the impact when the spreader 15 lands on the landing target surface 34.

The control system 20 can teach the operator an operation to avoid the contact by displaying the warning display 33g for avoiding the contact between the spreader 15 and the landing target surface 34 on the upper layer image 33. For example, when the warning display 33g is displayed in the upper layer image 33, if the operator lowers the spreader 15 as it is, the spreader 15 may come into contact with the container 31 existing around the landing target surface 34. Can be known. The operator can also know the operation of avoiding the contact by the warning display 33g. In this way, the operator can improve the safety in remote control by predicting the possibility of contact between the spreader 15 and the landing target surface 34 in advance.

The crane 10 includes a girder portion extending in one direction, a trolley supported by the girder portion and moving in the extending direction of the girder portion, and a spreader suspended from the trolley by a wire. It is not limited to the crane that handles the cargo of the container 31 for the ship. Examples of the crane include a gantry crane and an overhead crane that handle the cargo of the container with respect to the outpatient chassis and the storage lane at the container terminal.

In the above-described embodiment, in FIGS. 7 and 8, the y1 coordinate of the plane positions (x1 coordinate, y1 coordinate) of the spreader center point C2, the landing target surface center point C3, and the expected landing center point C4 is Although it is assumed that they are substantially the same, the y1 coordinate may be calculated sequentially in the same manner as the x1 coordinate. When the traveling device 16 moves the crane 10 in the y1 direction, the angle formed by the landing target and the expected landing angle with respect to the optical axis A1 in the front view are calculated, and the y2 coordinates in the laminated image 30 are aligned. Is also possible.

Similarly, the warning display 33g for urging the avoidance of contact between the spreader 15 and the landing target surface 34 other than the landing target surface 34 may also use a figure indicating movement in the y2 direction in the laminated image 30. The determination (S340) as to whether or not to display the warning display 33g is not limited to the above determination method. For example, based on the arrangement information of the containers 31 input to the control device 25, if the container 31 around the landing target surface 34 is lower than the landing target surface 34, it may be determined that there is no possibility of contact. ..

The control system 20 described above remotely controls the crane 10. By applying this control system 20, for example, the degree of overlap between the expected landing center point C4 and the landing target surface center point C3 is taken as a deviation, and based on the deviation, the trolley 13 and the spreader 15 are set by the control device 25. It is also possible to automate the operation of the crane 10 by the automatic operation.

Further, in the control system 20 described above, the position acquisition devices 23 and 24 measure the measured values rather than the time from when the camera 21 captures the unprocessed image 35 until the lower layer image 32 is displayed on the display device 22. An example is a system in which the time from acquisition to the display of the upper layer image 33 on the display device 22 is short. On the other hand, the control system 20 can also be applied to a system in which the lower layer image 32 is displayed on the display device 22 in a state where there is almost no delay time after the camera 21 takes an image. Even in such a system, the upper layer image 33 makes it possible to visually grasp the situation of the spreader 15 visually with figures, numerical values, and scales, which is advantageous for shortening the work time required for remote control and improving safety. Become.
In the control system 20 described above, the lower layer image 32 and the middle layer image 38 are displayed asynchronously, but the upper layer image 33 may be displayed in a state close to real time, and the lower layer image 32 and the middle layer image 38 are displayed in synchronization. You may let me.

10 Crane 11, 12 Girder 13 Trolley 14 Wire 15 Spreader 20 Control system 21 Camera 22 Display device 23, 24 Position acquisition device 25 Control device 30 Laminated image 32 Lower layer image 33 Upper layer image 34 Implantation target surface

Claims (10)

In a crane control system including a girder extending in one direction, a trolley supported by the girder and moving in the extending direction of the girder, and a hanger suspended from the trolley by a wire. ,
A camera installed on the trolley and sequentially capturing an image including a landing target surface on which the hanging tool and the hanging tool below the hanging tool lands, a display device for sequentially displaying the image captured by the camera, and the above. It has a position acquisition device for acquiring the plane positions of the hanger and the landing target surface, and a control device connected to the camera, the display device, and the position acquisition device.
Based on the plane position acquired by the position acquisition device, the control device displays a display on the transparent background that can identify that the hanger and the landing target surface are in a state of being aligned in a plan view. A configuration in which a drawn upper layer image is created, the created upper layer image and the image captured by the camera are superimposed and displayed on the display device asynchronously with each other, and the image is transmitted from the transparent background of the upper layer image. to,
The identifiable display includes a predicted landing mark indicating a predicted landing center point in which the center point of the hanging tool, which is the center of the hanging tool, is vertically projected onto a horizontal plane including the landing target surface in a plan view, and a plan view. A crane control system comprising a landing target mark indicating a landing target surface center point, which is the center of the landing target surface, and a landing target mark. In a crane control system including a girder extending in one direction, a trolley supported by the girder and moving in the extending direction of the girder, and a hanger suspended from the trolley by a wire. ,
A camera installed on the trolley and sequentially capturing an image including a landing target surface on which the hanging tool and the hanging tool below the hanging tool lands, a display device for sequentially displaying the image captured by the camera, and the above. It has a position acquisition device for acquiring the plane positions of the hanger and the landing target surface, and a control device connected to the camera, the display device, and the position acquisition device.
Based on the plane position acquired by the position acquisition device, the control device displays a display on the transparent background that can identify that the hanger and the landing target surface are in a state of being aligned in a plan view. A configuration in which a drawn upper layer image is created, the created upper layer image and the image captured by the camera are superimposed and displayed on the display device asynchronously with each other, and the image is transmitted from the transparent background of the upper layer image. to,
Further, based on the image, the control device creates an enlarged image obtained by enlarging a part of the image, and the created enlarged image is superimposed and displayed on the display device together with the upper layer image and the image. A crane control system characterized by the fact that it has been done. The lane control system according to claim 2 , wherein the enlarged image is an enlarged image of each of the four corners of the landing target surface. The upper layer created based on the plane position after the position acquisition device acquires the plane position, rather than the time from when the camera captures the image until the image is displayed on the display device. The crane control system according to any one of claims 1 to 3, wherein it takes a short time for an image to be displayed on the display device. The position acquisition device is configured to be capable of acquiring a three-dimensional position obtained by adding a height position to the plane position of the hanger and the landing target surface.
Claims 1 to 5 in which a display relating to the relative positional relationship between the crane and the landing target surface is drawn on the upper layer image by the control device based on the three-dimensional position acquired by the position acquisition device. The crane control system according to any one of 4. The condition around the landing target surface is preset in the control device, and the condition is set in advance.
A warning that the control device avoids contact between the hanging tool and other than the landing target surface based on the three-dimensional position acquired by the position acquisition device and the set surrounding conditions of the landing target surface. The crane control system according to claim 5 , wherein the display is drawn on the upper layer image. A notification device that is connected to the control device and emits sound or light is provided.
The control system for a crane according to claim 6 , wherein the control device causes the notification device to give a warning to avoid contact between the hanger and a surface other than the landing target surface. In the laminated image displayed in a laminated manner on the display device, the image is displayed in the entire display area of the display device, and the identifiable display displayed in a laminated manner on the upper layer of the image is arranged in the central portion of the display area. The crane control system according to any one of claims 1 to 7 , wherein the display other than the identifiable display is an image arranged on the outer edge of the display area. A crane suspended by a wire is lowered from a trolley that is supported by a girder extending in one direction and travels in the extending direction of the girder, and the lower end of the crane or a suspended load grasped by the crane. In the control method of the crane that makes the lower end of the landing on the landing target surface
The camera installed on the trolley sequentially captures images including the hanger and the landing target surface on which the hanger lands, and the position acquisition device captures the hanger and the landing target surface, respectively. Sequentially acquire the plane position of
As a display that can identify that the image captured by the camera and the landing target surface of the hanger and the landing target surface are aligned with each other in a plan view based on the acquired plane position by the control device . The expected landing mark indicating the expected landing center point where the center point of the hanger, which is the center of the hanger in plan view, is vertically projected onto the horizontal plane including the landing target surface, and the landing target surface in plan view. The landing target mark indicating the center point of the central landing target surface and the upper layer image created by drawing the display composed of the display on the transparent background are laminated and displayed on the display device asynchronously with each other, and the above-mentioned upper layer image is described. A method for controlling a crane, which comprises transmitting the image from a transparent background. A crane suspended by a wire is lowered from a trolley that is supported by a girder extending in one direction and travels in the extending direction of the girder, and the lower end of the crane or a suspended load grasped by the crane. In the control method of the crane that makes the lower end of the landing on the landing target surface
The camera installed on the trolley sequentially captures images including the hanger and the landing target surface on which the hanger lands, and the position acquisition device captures the hanger and the landing target surface, respectively. Sequentially acquire the plane position of
The control device transmits a display that can identify that the image captured by the camera and the landing target surface are aligned with each other in a plan view based on the acquired plane position. The upper layer image created by drawing on the background is superimposed and displayed on the display device asynchronously with each other, and the image is transmitted from the transparent background of the upper layer image.
Further, the control device is characterized in that a magnified image obtained by enlarging a part of the image is created based on the image, and the created magnified image is superimposed and displayed on the display device together with the upper layer image and the image. Crane control method.

Images