WO2019049511A1

WO2019049511A1 - Crane device

Info

Publication number: WO2019049511A1
Application number: PCT/JP2018/026546
Authority: WO
Inventors: 小林　雅人
Original assignee: 住友重機械搬送システム株式会社
Priority date: 2017-09-05
Filing date: 2018-07-13
Publication date: 2019-03-14
Also published as: CN111032561B; CN111032561A; JP6689467B2; JPWO2019049511A1; CN112938766A; CN112938766B

Abstract

A crane device comprises: a suspension means; an imaging unit that acquires a captured image by imaging under the suspension means; a distance information acquisition unit that acquires distance information to a plurality of measurement points within the range of the captured image of under the suspension means; a color information acquisition unit that acquires color information within the range of the captured image of under the suspension means; a display unit; and a control unit that processes the captured image and causes the same to be displayed on a display device. The control unit extracts, from the captured image, a subject area that includes an area in which an object of concern is located in the captured image on the basis of the distance information and the color information, detects a corresponding portion in the subject area that corresponds to the shape of a reference image stored in advance, and displays the captured image on the display unit while accentuating the detected corresponding portion.

Description

Crane equipment

One aspect of the present invention relates to a crane apparatus.

According to Patent Document 1, when a placed container is grasped by a spreader, or when a container grasped by a spreader is stacked on another placed container, acquisition is performed by a 3D camera provided in the spreader. The crane which detects the mounted container and acquires the positional relationship of a spreader and the said container is described based on the distance map.

Japanese Patent Application Publication No. 2015-533747

The 3D camera used in the crane as described above outputs measurement light to the area including the container to be detected, and detects the reflected light to obtain a distance map. For this reason, the 3D camera may erroneously detect that a container is present in an area where there is no container to be detected, for example, under the influence of strong sunlight or disturbance light outdoors. . In addition, the 3D camera may detect raindrops when used outdoors when it is raining, and may not properly detect a target object. Furthermore, in the 3D camera, for example, if there is a mirror surface portion in the container to be detected, the measurement light may be specularly reflected in a direction in which the reflected light can not be detected by the 3D camera. In this case, the 3D camera can not acquire the distance map for the mirror surface, and as a result, there is a possibility that the container can not be detected despite the presence of the container to be detected.

Then, one aspect of the present invention aims to provide a crane apparatus capable of more reliably acquiring the positional relationship between a lifting gear and an object.

A crane apparatus according to an aspect of the present invention includes: a lifting device that grips, holds, and unloads a load; an imaging unit that is provided on the lifting device and captures an image below the lifting device to obtain a captured image; A distance information acquisition unit provided on the lifting tool for acquiring distance information to a plurality of measurement points within the range of the captured image below the lifting tool, provided on the lifting tool, of the captured image A color information acquisition unit for acquiring color information in a range, a display unit for displaying a captured image, and a control unit for processing the captured image and displaying the image on the display unit, the control unit including distance information and color information Based on the target image including the region where the target is located in the captured image is extracted from the captured image, and a corresponding portion corresponding to the shape of the reference image stored in advance is detected in The image is highlighted on the display with the corresponding parts highlighted Make.

According to the crane apparatus according to an aspect of the present invention, in the captured image obtained by capturing the lower part of the lifting gear, the region where the object is located is estimated based on the distance information, and the object is located based on the color information The region of interest is estimated. Then, the apparatus extracts a target area including the area in which the target is located from the captured image based on these estimation results, and emphasizes and displays a corresponding portion corresponding to the shape of the reference image in the target area. Do. Therefore, this device can acquire the positional relationship between the lifting gear and the object using the distance information and the color information in a complementary manner. Therefore, since this device is not susceptible to, for example, the influence of disturbance light and the influence of specular reflection on an object, the positional relationship between the lifting gear and the object can be acquired more reliably.

In the crane apparatus according to an aspect of the present invention, the control unit may not detect the corresponding part outside the target area of the captured image. According to this, since it is only necessary to detect the corresponding part corresponding to the shape of the reference image for only the target area in the captured image, the time required for the process can be shortened.

In the crane apparatus according to one aspect of the present invention, the control unit estimates the first region in the captured image in which the target is located based on the distance information, and the target is in the captured image based on the color information. A second region located may be estimated, and a region including a region included in at least one of the first region and the second region may be extracted from the captured image as a target region. According to this, in addition to the area in which the object is estimated to be located by the distance information susceptible to the specular reflection at the object, the object is targeted by the color information which is not easily affected by the specular reflection at the object An area in which it is estimated that an object is located is also extracted from the captured image as a target area. Therefore, it is suppressed that an object can not be detected despite the presence of an object, and the positional relationship between the lifting gear and the object can be acquired more reliably.

In the crane apparatus according to one aspect of the present invention, the control unit estimates the first region in the captured image in which the target is located based on the distance information, and the target is in the captured image based on the color information. A second region located may be estimated, and a region including the region included in the first region and included in the second region may be extracted from the captured image as a target region. According to this, not only distance information susceptible to disturbance light but also color information unlikely to be affected by disturbance light is extracted from the captured image as a target region, an area estimated to be located by the object. Do. Therefore, it is suppressed that it will misdetect as a target object exists in the area | region where a target object does not exist, and it can acquire the positional relationship of a hanger and a target object more reliably.

The crane apparatus according to an aspect of the present invention includes a moving mechanism for moving the lifting gear, and the control unit causes the corresponding portion in the captured image to be captured and unloaded when the lifting gear performs loading and unloading of the load. The position shift of the reference portion where the corresponding portion is to be positioned may be detected, and the moving mechanism may move the hanger so as to eliminate the position shift. According to this, it is possible to accurately position the lifting device directly above the object.

In the crane apparatus according to an aspect of the present invention, the object may be at least a part of the load gripped by the lifting gear. According to this, in the situation where the load is gripped by the lifting tool, the positional relationship between the lifting tool and the load can be acquired more reliably.

In the crane apparatus according to an aspect of the present invention, the object may be at least a part of the mounting portion on which the load held by the lifting device is unloaded. According to this, in the situation where the load is unloaded by the lifting tool, the positional relationship between the lifting tool and the placement portion can be acquired more reliably.

According to one aspect of the present invention, it is possible to more reliably acquire the positional relationship between the lifting gear and the object.

FIG. 1 is a block diagram showing a crane apparatus according to the first embodiment. FIG. 2 is a front view of the crane apparatus. FIG. 3 is a side view of the crane apparatus. FIG. 4 is a perspective view showing a hanger and a coil. FIG. 5 is a view showing a captured image in the load grip control. FIG. 6 is a diagram showing a distance accuracy map in the load grip control. FIG. 7 is a diagram showing a color accuracy map in the load grip control. FIG. 8 is a diagram showing an OR probability map in the load grip control. FIG. 9 is a view showing a reference image. FIG. 10 is a view showing a highlighted image in the load gripping control. FIG. 11 is a diagram showing an AND probability map in the load grip control. FIG. 12 is a diagram showing a highlighted image in unloading control. FIG. 13 is a block diagram showing a crane apparatus according to a second embodiment. FIG. 14 is a front view of the crane apparatus. FIG. 15 is a perspective view of the crane apparatus. FIG. 16 is a diagram showing a reference image. FIG. 17 is a view showing a highlighted image in the load grip control. FIG. 18 is a view showing a highlighted image in unloading control.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and redundant description will be omitted.

First Embodiment
[Configuration of crane device]
FIG. 1 is a block diagram showing a crane apparatus 1 according to the first embodiment. FIG. 2 is a front view of the crane apparatus 1. FIG. 3 is a side view of the crane apparatus 1. FIG. 4 is a perspective view showing the hanger 30 and the coil C. As shown in FIG. As shown in FIGS. 1 to 4, the crane apparatus 1 according to the first embodiment is an overhead crane that transports a coil (load) C that is a steel plate wound in a cylindrical shape. The crane device 1 includes a moving mechanism 10, a lifting tool 30, an imaging unit 40, a distance information acquisition unit 41, a color information acquisition unit 42, a display unit 43, a control unit 50, and a cab 60. The crane apparatus 1 also includes a transport control unit 51 that controls the moving mechanism 10 and the lifting tool 30, and an encoder 31 connected to the transport control unit 51.

The coil C has a substantially cylindrical shape. At the axial center position of the coil C, a hanging hole H penetrating from one end to the other end of the coil C is formed. The coil C is placed on a skid (placement unit) S disposed in a matrix at predetermined intervals on a floor surface F of a building such as a factory or a warehouse in a direction in which the axial direction is horizontal. The coil C has a predetermined shape, a predetermined size, and a predetermined color. The outer peripheral surface of the coil C is mirror-like.

The moving mechanism 10 is a mechanism that moves the lifting tool 30 (horizontally, vertically, and rotationally). The moving mechanism 10 has a girder 12 and a trolley 13. The girder 12 supports the load of the trolley 13 and the hanger 30. The girder 12 is horizontally laid across both walls near the ceiling of the building. The girder 12 is movable in the horizontal direction orthogonal to the extending direction of the girder 12, and the trolley 13 and the hanger 30 can be moved in this direction.

The trolley 13 traverses the upper surface of the girder 12 along the extending direction of the girder 12. Thereby, the trolley 13 can move the hanger 30 along the extension direction of the girder 12. In addition, the trolley 13 has a hoisting mechanism 11 for winding up and down the hanging wire rope 17. Furthermore, the trolley 13 has a rotation mechanism (not shown) that rotationally moves the hanger 30 about the vertical axis in the vertical direction via the wire rope 17.

The hanger 30 is a device for gripping, holding, and unloading the coil C. The hanger 30 is hung on a wire rope 17 hanging down from the trolley 13. The lifting tool 30 is moved upward by winding up the wire rope 17 by the winding mechanism 11 of the trolley 13 and is moved downward by winding down the wire rope 17 by the winding mechanism 11. The hanger 30 has a base 18 and a pair of

opposed claws

19, 19. The wire rope 17 is connected to the upper surface side of the base portion 18, and the

claw portions

19, 19 are provided on the lower surface side so as to be able to open and close.

The toe of each of the

claws

19 and 19 is provided with a projection 20 projecting so as to face each other. By the

claws

19 and 19 closing, the respective convex portions 20 are inserted into the suspension holes H of the coil C from both sides. As a result, the hanger 30 holds the coil C. In the “state in which the hanger 30 holds the coil C”, the

inner surfaces

21 and 21 of the

claws

19 and 19 facing each other may or may not be in contact with the coil C.

The hanging tool 30 transfers the coil C in the following procedure. That is, the lifting tool 30 opens and closes the

claws

19 and 19 to grasp the load on the coil C placed on the skid S, and while holding the coil C, the moving mechanism 10 and the lifting mechanism 11 The coil C is transferred by being moved onto the skid S, and the

claws

19, 19 are opened and closed again to unload the coil C.

The encoder 31 is a sensor that detects the amount of operation for raising and lowering the hoisting mechanism 11 and the amount of movement of the trolley 13 in the transverse direction. Based on the detection result of the encoder 31, the conveyance control unit 51 acquires height information indicating the current height of the lifting tool 30. The encoder 31 may not detect the amount of movement of the trolley 13 in the transverse direction by detecting the amount of operation of raising and lowering the winding mechanism 11.

The transfer control unit 51 controls the driving of the crane device 1. For example, the transport control unit 51 moves the girder 12, traverses the trolley 13, rolls up and down the wire rope 17 by the winding mechanism 11 (moves the lifting tool 30 in the vertical direction), and rotates the lifting tool 30 by the rotation mechanism. It controls the rotational movement and the opening and closing of the

claws

19, 19. Further, the conveyance control unit 51 outputs, to the control unit 50, the height information of the hanger 30 obtained based on the detection result of the encoder 31.

The imaging unit 40 is an imaging device that is provided on the lifting tool 30 and captures an image of the lower side of the lifting tool 30 to obtain a captured image. The imaging unit 40 may be, for example, a camera. The imaging unit 40 is provided on the base 18 of the hanger 30 so as to face downward. A plurality of imaging units 40 may be provided on the hanger 30.

The distance information acquisition unit 41 is a device provided on the lifting tool 30 to obtain distance information to a plurality of measurement points within the range of the captured image captured by the imaging unit 40 below the lifting tool 30. The distance information acquisition unit 41 acquires distance information to each measurement point in association with the position in the captured image. The distance information acquisition unit 41 is not particularly limited as long as it is an apparatus capable of acquiring distance information, and may be, for example, a ToF [Time of Flight] camera, a 3D camera, or a 3D scanner. Good. The distance information acquisition unit 41 is provided in the vicinity of the imaging unit 40 in the base 18 of the hanging tool 30 so as to face downward. The distance information acquisition units 41 are provided by the number corresponding to the number of imaging units 40. The “distance information” is information indicating the distance between the distance information acquisition unit 41 and each measurement point. Here, the measurement points are, for example, points set on the floor F, the skid S, the upper surface of the coil C, etc., and may be set in a matrix, for example. The measurement point may be set to a predetermined relative position based on, for example, the position coordinate of the distance information acquisition unit 41 in the horizontal plane.

The color information acquisition unit 42 is a device provided on the hanging tool 30 to obtain color information within the range of the captured image captured by the imaging unit 40 below the lifting tool 30. The color information acquisition unit 42 acquires color information in association with the position in the captured image. The color information acquisition unit 42 is not particularly limited as long as it is an apparatus capable of acquiring color information, and may be, for example, a spectrum camera or a color camera. The color information acquisition unit 42 is provided in the vicinity of the imaging unit 40 in the base 18 of the hanger 30 so as to face downward. The color information acquisition units 42 are provided in the number corresponding to the number of imaging units 40. The “color information” is, for example, information which decomposes a color image into RGB signals and indicates it in vector representation.

The display unit 43 is a display device that displays a captured image. Further, the display unit 43 displays the emphasized image generated by the control unit 50. The display unit 43 may be, for example, a display. The display unit 43 is provided, for example, in the driver's cab 60.

The control unit 50 processes the captured image to generate an emphasized image, and causes the display unit 43 to display the generated emphasized image (the details will be described later). The control unit 50 is configured to unload the coil C held by the lifting tool 30 onto the skid S by unloading the coil C held by the lifting tool 30. It is possible to generate an enhanced image in the down control.

[Load control]
Hereinafter, the display of the emphasis image in the load grip control will be described.

FIG. 5 is a view showing a captured image P1 in the load grip control. As shown in FIG. 5, the control unit 50 causes the display unit 43 to display the captured image P1 captured by the imaging unit 40. In FIG. 5, the coil C placed below the hanger 30, one claw portion 19 of the hanger 30, and the skid S on which the coil C is not placed are displayed. In FIG. 5, the hanger 30 is at a position offset from immediately above the mounted coil C. In FIG. 5, the coil C is placed on a skid S (not shown) below the coil C. Since the outer peripheral surface of the coil C is mirror-like, the coil C in the captured image P1 has a reflection portion R in which the surrounding scene is reflected.

Subsequently, the control unit 50 extracts a target area including the area in which the target is located in the captured image P1 from the captured image P1 as follows. Here, the “target object” is the coil C gripped by the lifting tool 30.

FIG. 6 is a diagram showing the distance accuracy map P2 in the load grip control. As shown in FIG. 6, the distance information acquisition unit 41 corresponds distance information to a plurality of measurement points (not shown) in the range of the captured image P1 below the lifting tool 30 with the position in the captured image P1. Get it. As an example, the measurement points may be set at positions where predetermined numbers (for example, 20) in the vertical direction of the captured image P1 and predetermined numbers (for example, 30) in the horizontal direction are arranged in a matrix at equal intervals.

The control unit 50 estimates, based on the distance information acquired by the distance information acquisition unit 41, a first area A1 in which the coil C, which is an object in the captured image P1, is located. More specifically, on the premise that the coil C has a predetermined shape and a predetermined size, the control unit 50 compares the lifting device 30 and the coil C with each other based on the current height information of the lifting device 30. Predict the current distance between. Then, the control unit 50 compares the predicted distance with the distance related to the distance information acquired by the distance information acquisition unit 41 to estimate the first area A1 in which the coil C is located in the captured image P1. For example, if the difference between the predicted distance and the distance related to the distance information acquired by the distance information acquisition unit 41 is within a predetermined range, the control unit 50 determines that the coil C is located in that area, If it is out of the predetermined range, it may be determined that the coil C is not located in the region.

In FIG. 6, the control unit 50 appropriately determines that the area in which the coil C is located is in the vicinity of the axial center of the coil C in a plan view (white area in FIG. 6). On the other hand, in a region (gray region in FIG. 6) far from the axial center of the coil C in a plan view, the measurement light of the distance information acquisition unit 41 is incident at an incident angle close to parallel to the outer peripheral surface of the mirror C. It is incident. Therefore, in the area, the distance information acquisition unit 41 can not detect the reflected light, and as a result, the control unit 50 erroneously determines that the coil C is not located.

FIG. 7 is a diagram showing the color accuracy map P3 in the load grip control. As shown in FIG. 7, the distance information acquisition unit 41 acquires color information within the range of the captured image P1 below the lifting tool 30 in association with the position in the captured image P1. The control unit 50 estimates, based on the color information acquired by the color information acquisition unit 42, the second area A2 in which the coil C, which is an object in the captured image P1, is located. More specifically, on the premise that the coil C has a predetermined color, the control unit 50 positions the coil C based on the degree of color correlation with the predetermined color in the captured image P1. The second area A2 is estimated. In addition, a known method can be used as a method of determining "the degree of color correlation". For example, the control unit 50 converts the color information into coordinates on the chromaticity diagram, and evaluates the distance between the coordinates on the chromaticity diagram and the coordinates on the chromaticity diagram of the coil C stored in advance. The degree of color correlation may be determined by a method. The control unit 50 stores the color of the coil C in advance. The control unit 50 determines that the coil C is positioned in the area if the degree of color correlation is equal to or higher than a predetermined value, and determines that the coil C is not positioned in the area if it is less than the predetermined value. You may

In FIG. 7, the control unit 50 appropriately determines that the coil C is located in a portion (white area in FIG. 7) of the coil C excluding the reflected portion R in a plan view. It is judged. On the other hand, since the reflection portion R has a color which is largely different from the color of the coil, the control unit 50 erroneously determines that the coil C is not located.

FIG. 8 is a diagram showing an OR probability map P4 in the load grip control. The OR accuracy map P4 shown in FIG. 8 shows an OR region A3 (white part in FIG. 8) included in at least one of the first region A1 of the distance accuracy map P2 and the second region A2 of the color accuracy map P3. There is. The control unit 50 sets an area including the OR area A3 in the OR probability map P4 (for example, an area obtained by enlarging the outer edge of the OR area A3 by a predetermined width) as the target area A4. As described above, the control unit 50 extracts, from the captured image P1, a target region A4 which is a region in which the coil C is located in the captured image P1.

Subsequently, the control unit 50 detects a corresponding portion corresponding to the shape of the reference image stored in advance in the target area A4 of the captured image P1 as follows.

FIG. 9 is a view showing a reference image P5. FIG. 10 is a view showing a highlight image P6 in the load grip control. As shown in FIGS. 9 and 10, the control unit 50 stores in advance a shape (specifically, a plan view of the coil C) in the captured image P1 of the coil C which is an object as a reference image P5. There is. Then, the control unit 50 detects a corresponding portion A5 corresponding to the shape of the reference image P5 in a region corresponding to the target region A4 in the captured image P1 using a known image recognition method. The "corresponding to the shape of the reference image P5" may have a size different from that of the reference image P5, or may be a shape in which the reference image P5 is inclined around an axis in the vertical direction. As the “image recognition method”, for example, pattern matching, machine learning, etc. may be used. Note that the control unit 50 does not detect the corresponding part A5 outside the target area A4 in the captured image P1.

Subsequently, the control unit 50 generates an emphasized image P6 in which the corresponding part A5 is emphasized in the captured image P1, and causes the display unit 43 to display the generated emphasized image P6. Here, with respect to the captured image P1, the emphasis image P6 displays the target area A4 in color and displays areas other than the target area A4 in gray. Further, in the emphasized image P6, the corresponding part A5 is surrounded by a broken line. In the emphasized image P6, the corresponding part A5 is not limited to being surrounded by a broken line as long as it is emphasized so as to be easily recognized by the operator, and may be displayed in a predetermined color, for example. It is also good, it may be cut out.

Next, correction | amendment of position shift of the hanger 30 with respect to the coil C which is a target object is demonstrated.

As shown in FIG. 10, the control unit 50 determines, based on the detection result of the encoder 31, a reference portion A6 at which the corresponding portion A5 should be positioned in the captured image P1 when the hanger 30 grips the load of the coil C. Calculated from height information acquired. The reference portion A6 is, for example, the shape of the coil C in a plan view in the case where the coil C is placed on the skid S directly under the lifting tool 30 (downward in the vertical direction). The control unit 50 detects positional deviation between the corresponding portion A5 and the reference portion A6 in the captured image P1 (or the enhanced image P6). Further, the control unit 50 may detect the relative distance between the lifting tool 30 and the coil C in the height direction based on the magnification of the size of the corresponding part A5 and the reference part A6, and the corresponding part A5 and the reference part A shift in the rotational direction between the hanger 30 and the coil C may be detected based on the angular shift with A6. The control unit 50 outputs information on the detected positional deviation to the conveyance control unit 51.

The transport control unit 51 causes the moving mechanism 10 to move the hanger 30 so as to eliminate the positional deviation, based on the information on the positional deviation input from the control unit 50. That is, the moving mechanism 10 moves the hanger 30 immediately above the coil C by moving the hanger 30 such that the positional deviation is eliminated by the movement of the girder 12 and the traversing of the trolley 13. At this time, the lifting mechanism 30 may be rotationally moved about the vertical axis by driving the rotation mechanism as needed.

The method using the OR probability map P4 has been described above for the display of the enhanced image P6 in the load grip control by the control unit 50. The control unit 50 can also execute a method using an AND certainty map instead of the OR certainty map P4 for the display of the emphasized image P6 in the grip control.

FIG. 11 is a view showing an AND probability map P7 in the load grip control. The AND probability map P7 shown in FIG. 11 is the AND region A7 (white part in FIG. 11) included in the first region A1 of the distance probability map P2 and included in the second region A2 of the color probability map P3. It shows. The control unit 50 may set an area including the AND area A7 in the AND likelihood map P7 (for example, an area obtained by expanding the outer edge of the AND area A7 by a predetermined width) as the target area. As described above, the control unit 50 may extract, from the captured image P1, a target region which is a region in which the coil C is located in the captured image P1. Note that whether to use the OR probability map P4 or the AND probability map P7 is appropriately selected according to the object, for example.

[Unloading control]
The control unit 50 can display a highlighted image also in the unloading control by processing the captured image in the same manner as the load grip control. Hereinafter, the display of the emphasized image in the unloading control will be described.

FIG. 12 is a view showing a emphasized image P8 in unloading control. The highlighted image P8 in FIG. 12 includes the coil C, the claw portion 19 of the hanger 30 holding the coil C, and a part of the skid S to which the coil C is unloaded. The skid S has a loading mark M having a predetermined shape, a predetermined size, and a predetermined color. In the emphasized image P8, the lifting tool 30 is at a position offset from immediately above the skid S to which the coil C is unloaded, and is tilted about the vertical axis with respect to the skid S. Here, the “object” is the stowing mark M of the skid S to which the coil C held by the hanger 30 is unloaded. The shape and size of the stowage mark M is the shape and size of a virtual columnar portion extending from the entire stowage mark M to the floor surface F in the vertical direction among the skids S having the stowage mark M. is there.

The highlighted image P8 displays the target area A8 including the OR area in the OR probability map in color and grays out the area other than the target area A8 with respect to the captured image before unloading the coil C on the skid S. doing. The target area A8 may be set based on the AND area in the AND likelihood map instead of the OR area in the OR likelihood map. In the enhanced image P8, the area of the stowing mark M, which is the corresponding portion A9 detected in the target area A8, is surrounded by a broken line. Further, in the emphasized image P8, the reference portion A10 where the corresponding portion A9 should be positioned when unloading the coil C is performed by the lifting tool 30 is indicated by a dashed dotted line. The reference portion A10 is a shape of the stowage mark M in a plan view in the case where the stowing mark M is located immediately below the hanger 30 (downward in the vertical direction).

The control unit 50 detects positional deviation between the corresponding part A9 and the reference part A10. Further, the control unit 50 may detect the relative distance between the lifting tool 30 and the coil C in the height direction based on the magnification of the size of the corresponding part A9 and the reference part A10, and the corresponding part A9 and the reference part A shift in the rotational direction between the hanger 30 and the coil C may be detected based on the angular shift with A10. The control unit 50 outputs information on the detected positional deviation to the conveyance control unit 51. The transport control unit 51 causes the moving mechanism 10 to move the hanger 30 so as to eliminate the positional deviation, based on the information on the positional deviation input from the control unit 50. That is, the moving mechanism 10 moves the lifting tool 30 immediately above the skid S by moving the lifting tool 30 so that the positional deviation is eliminated by the movement of the girder 12, the traversing of the trolley 13, and the drive of the rotation mechanism. Let

[Action and effect]
As described above, according to the crane device 1, in the captured image P1 obtained by capturing the lower part of the lifting tool 30, the first region A1 in which the coil C or the stowage mark M, which is the object, is located based on the distance information. Is estimated, and the second region A2 in which the target object coil C or stowing mark M is located is estimated based on the color information. Then, the crane apparatus 1 extracts target areas A4 and A8 including the area in which the target is located from the captured image P1 based on the estimation results, and corresponds to the shape of the reference image in the target areas A4 and A8. The corresponding parts A5 and A9 are highlighted and displayed. Therefore, the crane apparatus 1 can acquire the positional relationship of the lifting gear 30 and a target object using distance information and color information complementarily. Therefore, since the crane apparatus 1 is hard to receive the influence of disturbance light, and the influence of the specular reflection in a target object, it can acquire the positional relationship of the hanger 30 and a target object more reliably.

In the crane device 1, the control unit 50 does not detect the corresponding parts A5 and A9 outside the target areas A4 and A8 of the captured image P1. As a result, it is only necessary to detect corresponding parts A5 and A9 corresponding to the shape of the reference image for only the target areas A4 and A8 in the captured image P1, so that the time required for processing can be shortened.

In the crane device 1, the control unit 50 estimates the first area A1 in which the coil C or stowing mark M is located in the captured image P1 based on the distance information, and based on the color information, A region including the OR region included in at least one of the first region A1 and the second region A2 by estimating the second region A2 in which the coil C or the stowage mark M is located in the captured image P1. Are extracted from the captured image P1 as target areas A4 and A8. Thus, in addition to the first area A1 in which the object is estimated to be located by the distance information susceptible to the specular reflection of the object, the color information which is less susceptible to the specular reflection of the object The second area A2 in which the object is estimated to be located is also extracted from the captured image P1 as the target areas A4 and A8. Therefore, it is suppressed that an object can not be detected despite the presence of the object, and the positional relationship between the hanging tool 30 and the object can be acquired more reliably.

Alternatively, in the crane device 1, the control unit 50 estimates the first area A1 in which the coil C or the stowage mark M is the target in the captured image P1 based on the distance information, and the color information is used. To estimate the second region A2 in which the coil C or the stowage mark M is located in the captured image P1, and the AND region included in the first region A1 and included in the second region A2 The areas to be included are extracted from the captured image P1 as target areas A4 and A8. Thus, not only distance information susceptible to disturbance light but also color information unlikely to be affected by disturbance light is assumed to be an area in which the object is located as a target area A4 or A8. Extract from Therefore, erroneous detection as the presence of the object is suppressed even in the region where the object does not exist, and the positional relationship between the lifting tool 30 and the object can be acquired more reliably.

The crane apparatus 1 includes the moving mechanism 10 for moving the lifting tool 30, and the control unit 50 grips or unloads the corresponding portions A5 and A9 in the captured image P1 and the coil C whose load is the lifting tool 30. In this case, the positional deviation of the reference parts A6 and A10 where the corresponding parts A5 and A9 are to be positioned in the captured image P1 is detected, and the moving mechanism 10 moves the hanger 30 so as to eliminate the positional deviation. Thereby, the hanger 30 can be accurately positioned immediately above the object.

In the crane apparatus 1, the object is a coil C gripped by the lifting tool 30. Thereby, in the situation where the load on the coil C is gripped by the hanger 30, the positional relationship between the hanger 30 and the coil C can be acquired more reliably.

Alternatively, in the crane apparatus 1, the object is the stowing mark M of the skid S to which the coil C held by the hanger 30 is unloaded. Thereby, in the condition of unloading the coil C by the hanger 30, the positional relationship between the hanger 30 and the skid S can be acquired more reliably.

Second Embodiment
[Configuration of crane device]
The crane apparatus which concerns on 2nd Embodiment is demonstrated. The crane apparatus according to the second embodiment is different from the crane apparatus 1 according to the first embodiment in the type of crane and the load to be transferred. Hereinafter, points different from the crane apparatus 1 according to the first embodiment will be mainly described.

FIG. 13 is a block diagram showing a crane apparatus 1A according to a second embodiment. FIG. 14 is a front view of the crane apparatus 1A. FIG. 15 is a perspective view of the crane apparatus 1A. As shown in FIGS. 13 to 15, the crane apparatus 1A according to the second embodiment is, for example, in a container yard Y at a container terminal where transfer of a container (load) D, etc. is performed to a container ship that has been docked. It is a container handling crane which is disposed and performs loading and unloading of the container D. The crane apparatus 1A includes a moving mechanism 10A, a hanger 30A, an imaging unit 40, a distance information acquisition unit 41, a color information acquisition unit 42, a display unit 43, a control unit 50, and a cab 60A. The hanger 30A is also referred to as a spreader. In addition, the crane apparatus 1A includes a transport control unit 51A that controls the moving mechanism 10A and the hanging tool 30A, and an encoder 31 and a shake sensor 32 connected to the transport control unit 51A.

The container D is a container such as an ISO standard container. The container D is in the form of a long rectangular parallelepiped and has a predetermined length of, for example, 20 feet or 40 feet in the longitudinal direction. The container D is provided with the to-be-locked part G in which the hole Gh was each formed in the four corners of the upper surface (refer FIG. 16). The container D is stacked in one or more stages in a container yard Y to form a plurality of rows E. The rows E are vertically and horizontally aligned such that the longitudinal direction of the containers D constituting the rows E is parallel to the longitudinal direction of the containers D constituting the other rows E. The container D has a predetermined shape and a predetermined size. In addition, the locked portion G has a predetermined shape, a predetermined size, and a predetermined color.

In the following description, among the containers D, the container D which is about to be gripped by the hanger 30A is referred to as a target container D1 (see FIG. 14). In addition, the container D held by the hanger 30A is referred to as a holding container D2, and the container D on which the holding container D2 is about to be unloaded is referred to as a target container (placement unit) D3 (see FIG. 15). . The target container D1, the holding container D2, and the target container D3 are such that how to call the container D changes according to the transfer status of the container D.

The moving mechanism 10A is a mechanism that moves the lifting tool 30A (horizontally, vertically, and rotationally). The moving mechanism 10A includes a traveling device 14, two pairs of

legs

15, 15, a girder 12A, and a trolley 13A. The traveling device 14 includes tire wheels provided at lower ends of the two pairs of

leg portions

15 and 15, and driving the tire wheels by the traveling motor causes two pairs of leg portions 15, It is possible to run 15 back and forth. The girder 12A is horizontally laid over the upper end portions of the two pairs of

leg portions

15 and 15 substantially horizontally. As a result, the girder 12A can move in the horizontal direction orthogonal to the extending direction of the girder 12A by causing the traveling device 14 to travel the two pairs of

legs

15 and 15 forward and backward. The trolley 13A and the hanger 30A can be moved in this direction.

The trolley 13A traverses the upper surface of the girder 12A along the extending direction of the girder 12A. Thereby, the trolley 13A can move the hanger 30A along the extension direction of the girder 12A. In addition, the trolley 13A has a hoisting mechanism 11A that winds up and down the hanging wire rope 17A. Furthermore, the trolley 13A has a rotation mechanism (not shown) that rotationally moves the hanger 30A around an axis in the vertical direction via the wire rope 17A.

Under the girder 12A, a traveling path W of a transport carriage V such as a trailer or an AGV (Automated Guided Vehicle) is laid. The crane apparatus 1A grips the container D carried in by the transport carriage V, and the container D is placed on the container yard Y or another container D (target container D3) placed on the container yard Y. Unload on top. In addition, the crane apparatus 1A grips the container D (target container D1) placed on the container yard Y or another container D placed on the container yard Y, and moves the container D The container D is unloaded onto the transport carriage V, and the container D is unloaded by the transport carriage V to the outside.

The hanger 30A is a device for gripping, holding, and unloading the container D. The hanger 30A holds the container D from the upper surface side. The hanger 30A is suspended on a wire rope 17A suspended from the trolley 13A. The lifting gear 30A moves upward by winding up the wire rope 17A by the winding mechanism 11A of the trolley 13A, and moves downward by rolling down the wire rope 17A by the winding mechanism 11A. The hanger 30A has a main body 18A and four lock pins (not shown).

The main body portion 18A has a shape and a size corresponding to the shape and the size of the container D in a plan view. That is, the main body portion 18A has a long rectangular shape in plan view. The body portion 18A includes a sheave 22 around which the wire rope 17A is wound, on the upper surface side of the central portion in the longitudinal direction.

The lock pin is a mechanism for holding the container D. The lock pins are provided at the four corners of the lower surface of the main body portion 18A so as to project downward from the main body portion 18A. The lock pin is provided at a position corresponding to the hole Gh of the engaged portion G of the container D when the hanger 30A holds the container D. The lock pin is, for example, a twist lock pin, and includes at its lower end a locking piece rotatable about a vertical axis. The lock pins are engaged with the container D by entering holes Gh of the engaged portions G provided at the four corners of the upper surface of the container D and rotating the locking pieces by 90 degrees.

The encoder 31 is a sensor that detects the amount of operation of raising and lowering of the winding mechanism 11 and the amount of movement of the trolley 13A in the transverse direction. Based on the detection result of the encoder 31, the transport control unit 51A acquires height information indicating the current height of the hanging tool 30A. The encoder 31 may not detect the amount of movement of the trolley 13 </ b> A while detecting the amount of operation of raising and lowering of the winding mechanism 11. The shake sensor 32 is a sensor that detects the amount of shake of the hanger 30A due to the wire rope 17A swinging. Based on the detection result of the shake sensor 32, the conveyance control unit 51A acquires shake amount information indicating the current shake amount of the lifting tool 30A.

The transfer control unit 51A controls the drive of the crane device 1A. For example, the transport control unit 51A moves the girder 12A, traverses the trolley 13A, rolls up and down the wire rope 17 by the winding mechanism 11A (moves the lifting device 30A in the vertical direction), and rotates the lifting device 30A by the rotation mechanism. It controls the rotational movement and the rotation of the locking piece of the lock pin. Further, the conveyance control unit 51A controls the height information of the lifting tool 30A acquired based on the detection result of the encoder 31 and the deflection amount information of the lifting tool 30A acquired based on the detection result of the shake sensor 32 as the control unit 50. Output to

The imaging unit 40 is an imaging device that is provided on the hanging tool 30A and captures an image of the lower side of the hanging tool 30A to obtain a captured image. The configuration and function of the imaging unit 40 are the same as those described above in the first embodiment.

The distance information acquisition unit 41 is a device provided in the hanging tool 30A to obtain distance information to a plurality of measurement points within the range of the captured image captured by the imaging unit 40 below the lifting tool 30A. Here, the measurement points are, for example, points set on the upper surface of the container yard Y, the container D mounted on the container yard Y, and the like, and may be set, for example, in a matrix. The measurement point may be set to a predetermined relative position based on, for example, the position coordinate of the distance information acquisition unit 41 in the horizontal plane. The configuration and function of the distance information acquisition unit 41 are the same as those described above in the first embodiment.

The color information acquisition unit 42 is a device that is provided to the hanging tool 30A and acquires color information within the range of the captured image captured by the imaging unit 40 below the lifting tool 30A. The configuration and function of the color information acquisition unit 42 are the same as those described above in the first embodiment.

The display unit 43 is a display device that displays a captured image. Further, the display unit 43 displays the emphasized image generated by the control unit 50. The display unit 43 may be, for example, a display. The display unit 43 is provided, for example, in the driver's cab 60A.

The control unit 50 processes the captured image to generate a emphasized image, and causes the display unit 43 to display the generated emphasized image. The control unit 50 can generate an enhanced image in load gripping control in which the object container D1 is gripped by the lifting implement 30A, and an enhanced image in unloading control in which the holding container D2 is unloaded onto the target container D3.

[Load control]
Hereinafter, the display of the emphasis image in the load grip control will be described. The control unit 50 displays the enhanced image in the cargo hold control in the second embodiment by processing the captured image in the same manner as the cargo hold control in the first embodiment.

FIG. 16 is a diagram showing a reference image P9. FIG. 17 is a diagram showing a highlight image P10 in the load grip control. As shown in FIGS. 16 and 17, the control unit 50 stores in advance the shape of the locked portion G of the container D as a reference image P9. The highlighted image P10 in FIG. 17 includes a part of the main body 18A of the hanger 30A and the vicinity of the engaged portion G of the target container D1 which is about to be gripped by the hanger 30A. In the emphasized image P10, the hanger 30A is at a position offset from immediately above the target container D1. Here, the “target object” is the locked portion G of the target container D1.

The control unit 50 estimates, based on the distance information acquired by the distance information acquisition unit 41, the first region in which the to-be-locked portion G of the target container D1 which is the target object is located. In addition, the control unit 50 estimates a second region in which the to-be-locked portion G of the target container D1 that is the target object is located, based on the color information acquired by the color information acquisition unit 42. The control unit 50 sets an area including the OR area included in at least one of the first area and the second area as the target area A11. As described above, the control unit 50 extracts the target area A11 from the captured image.

Then, the control unit 50 detects, in the target area A11, a corresponding portion corresponding to the shape of the reference image stored in advance using a known image recognition method. The control unit 50 does not detect the corresponding part A12 outside the target area A11 in the captured image. The control unit 50 generates an emphasized image P10 in which the corresponding part A12 detected in the captured image is emphasized, and causes the display unit 43 to display the generated emphasized image P10.

The highlighted image P10 displays the target area A11 in color and the areas other than the target area A11 in gray with respect to the captured image before the target container D1 is grasped. The target area A11 may be set based on the AND area in the AND likelihood map instead of the OR area in the OR probability map. In the emphasized image P10, the area of the locked portion G which is the corresponding part A12 detected in the target area A11 is surrounded by a broken line. Further, in the emphasized image P10, a reference portion A13 where the corresponding portion A12 should be positioned when the lifting tool 30A grips the target container D1 is indicated by a dashed-dotted line. The reference portion A13 is a shape of the locked portion G in a plan view when the locked portion G is positioned directly below the hanging tool 30A (downward in the vertical direction).

The control unit 50 detects positional deviation between the corresponding part A12 and the reference part A13. In addition, the control unit 50 may detect the relative distance between the lifting tool 30 and the target container D1 in the height direction based on the magnification of the size of the corresponding part A12 and the reference part A13, and the reference part A12 and the reference A shift in the rotational direction between the hanger 30 and the target container D1 may be detected based on the angular shift with the portion A13. The control unit 50 outputs information on the detected positional deviation to the conveyance control unit 51A. The transport control unit 51A causes the moving mechanism 10A to move the hanger 30A based on the information on the positional deviation input from the control unit 50 so that the positional deviation is eliminated. That is, the moving mechanism 10A moves the hanger 30A to the position immediately above the target container D1 by moving the hanger 30A such that the positional deviation is eliminated by the movement of the girder 12A and the traversing of the trolley 13A. At this time, the lifting mechanism 30A may be rotationally moved about an axis in the vertical direction by driving the rotation mechanism as needed.

FIG. 18 is a view showing a highlighted image P11 in unloading control. In the highlight image P11 of FIG. 18, a part of the main body 18A of the hanging member 30A, a part of the holding container D2 held by the hanging member 30A, and a target container D3 to which the holding container D2 is to be unloaded. The vicinity of the to-be-locked portion G is included. In the emphasized image P11, the hanger 30A is at a position deviated from immediately above the target container D3. Here, the “object” is the locked portion G of the target container D3.

The highlighted image P11 displays the target area A11, which is an area including the OR area in the OR probability map, in color with respect to the captured image before unloading the holding container D2 onto the target container D3, and also displays a color other than the target area A11. The area is displayed in gray. The target area A11 may be set based on the AND area in the AND likelihood map instead of the OR area in the OR probability map. In the emphasized image P11, the area of the locked portion G which is the corresponding part A12 detected in the target area A11 is surrounded by a broken line. Further, in the emphasized image P11, a reference portion A13 where the corresponding portion A12 should be positioned when the lifting tool 30A unloads the holding container D2 is displayed by a dashed-dotted line. The reference portion A13 is a shape of the locked portion G in a plan view when the locked portion G is positioned directly below the hanging tool 30A (downward in the vertical direction).

The control unit 50 detects positional deviation between the corresponding part A12 and the reference part A13. Further, the control unit 50 may detect the relative distance between the lifting tool 30 and the target container D3 in the height direction based on the magnification of the size of the corresponding part A12 and the reference part A13, and the reference part A12 and the reference A shift in the rotational direction between the hanger 30 and the target container D3 may be detected based on the angular shift with the part A13. The control unit 50 outputs information on the detected positional deviation to the conveyance control unit 51A. The transport control unit 51A causes the moving mechanism 10A to move the hanger 30A based on the information on the positional deviation input from the control unit 50 so that the positional deviation is eliminated. That is, the moving mechanism 10A moves the hanger 30A to the position immediately above the target container D3 by moving the hanger 30 so that the positional deviation is eliminated by the movement of the girder 12A and the traversing of the trolley 13A. At this time, the lifting mechanism 30A may be rotationally moved about an axis in the vertical direction by driving the rotation mechanism as needed.

In the unloading control, since the hanger 30A holds the holding container D2, when the holding container D2 descends toward the target container D3, the engaged portion G of the target container D3 is a holding container. There is a case where it is hidden in D2 and is not displayed on the captured image. Therefore, the crane device 1A shifts the positional deviation between the corresponding portion A12 and the reference portion A13 in advance in a state where the lifting tool 30A is positioned at a predetermined height before the locked portion G of the target container D3 is not displayed. Is detected, and the detected value of the shake sensor 32 is calibrated based on the detection result, so that the detected value of the shake sensor 32 is associated with the positional deviation. Thereafter, the crane device 1A monitors the state of positional deviation between the corresponding portion A12 and the reference portion A13 based on the detection result of the shake sensor 32, while lowering the holding container D2 toward the target container D3 by the lifting tool 30A. . Thereby, crane apparatus 1A may maintain the state where hanger 30A is located directly on target container D3.

[Action and effect]
As described above, according to the crane apparatus 1A, in the captured image obtained by capturing the lower part of the lifting tool 30A, the locked portion G of the target container D1 or the target container D3 that is the target object is locked based on the distance information. The first region in which the portion G is located is estimated, and the locked portion G of the target container D1 that is the target object or the locked portion G of the target container D3 is positioned based on the color information An area is estimated. Then, based on these estimation results, the crane apparatus 1A extracts a target area A11 including the area in which the target is located from the captured image, and a corresponding portion A12 corresponding to the shape of the reference image P9 in the target area A11. Emphasize and display. Therefore, the crane apparatus 1A can acquire the positional relationship between the hanger 30A and the object using the distance information and the color information in a complementary manner. Therefore, the crane device 1A is less susceptible to the effects of disturbance light, the influence of specular reflection on the object, and the detection of raindrops, for example, so that the positional relationship between the lifting gear 30A and the object is acquired more reliably. Can.

In the crane apparatus 1A, the control unit 50 does not detect the corresponding part A12 outside the target area A11 of the captured image. As a result, it is only necessary to detect the corresponding part A12 corresponding to the shape of the reference image P9 only in the target area A11 of the captured image, so that the time required for processing can be shortened.

In the crane apparatus 1A, the control unit 50 is a first region in which the to-be-locked portion G of the target container D1 or the to-be-locked portion G of the target container D3 is the object in the captured image based on the distance information. To estimate a second region in which the to-be-locked portion G of the target container D1 as the object or the to-be-locked portion G of the target container D3 is located in the captured image based on the color information A region including the OR region included in at least one of the region and the second region is extracted from the captured image as a target region A11. Thereby, in addition to the first region estimated to be located by the influence of disturbance light and distance information susceptible to the detection of raindrops, color information which is not easily affected by specular reflection at the object The second area, which is estimated to have the object located by the above, is also extracted from the captured image as the target area A11. Therefore, it is suppressed that an object can not be detected despite the presence of an object, and the positional relationship between the hanger 30A and the object can be acquired more reliably.

Alternatively, in the crane device 1A, the control unit 50 is configured such that the locked portion G of the target container D1 that is the target object or the locked portion G of the target container D3 is located in the captured image based on the distance information. 1 area is estimated, and a second area in which the to-be-locked portion G of the target container D1 as the object or the to-be-locked portion G of the target container D3 is located in the captured image is estimated based on color information; A region including the AND region included in the first region and included in the second region is extracted from the captured image as a target region A11. As a result, an area in which the object is estimated to be located not only by distance information susceptible to disturbance light but also by color information not easily affected by disturbance light is extracted as a target area A11 from the captured image. . Therefore, erroneous detection as the presence of the target is suppressed even in the area where the target does not exist, and the positional relationship between the hanging tool 30A and the target can be acquired more reliably.

The crane apparatus 1A includes a moving mechanism 10A for moving the lifting tool 30A, and the control unit 50 performs loading or unloading of the corresponding part A12 in the captured image and the container D in which the lifting tool 30A is a load. The positional deviation of the reference part A13 where the corresponding part A12 should be positioned in the captured image is detected, and the moving mechanism 10A moves the hanger 30A so that the positional deviation disappears. Thereby, the hanger 30A can be accurately positioned immediately above the object.

In the crane apparatus 1A, the target object is the to-be-locked portion G of the target container D1 gripped by the lifting tool 30A. Thereby, in a situation where the target container D1 is gripped by the lifting tool 30A, the positional relationship between the lifting tool 30A and the target container D1 can be acquired more reliably.

Alternatively, in the crane device 1A, the target object is the engaged portion G of the target container D3 to which the holding container D2 held by the lifting tool 30A is unloaded. As a result, in the situation where the holding container D2 is unloaded by the hanger 30A, the positional relationship between the hanger 30A and the target container D3 can be acquired more reliably.

[Modification]
The embodiments described above can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art.

For example, in the first embodiment and the second embodiment, two or all of the devices constituting the imaging unit 40, the distance information acquisition unit 41, and the color information acquisition unit 42 are common devices. It is also good. For example, the imaging unit 40 and the color information acquisition unit 42 may be configured by a common color camera, and the imaging unit 40, the distance information acquisition unit 41, and the color information acquisition unit 42 are common RGB image integrated TOF cameras It may be configured.

In the first and second embodiments, when an object is detected in a region where an object should not exist based on the OR accuracy map or the AND accuracy map, the control unit 50 sets an obstacle for the object. It may be recognized as

In addition, the object may have a predetermined shape, a predetermined size, and a predetermined color. For example, in the first embodiment, the object may be other than the loading mark M of the coil C or the skid S. Alternatively, the object may be a part of the coil C, or the entire skid S. In the second embodiment, the object may be something other than the locked portion G of the target container D1 or the locked portion G of the target container D3. Alternatively, the object may be the entire target container D1 or the entire target container D3.

In addition, the control unit 50 may store reference portions A6, A10, and A13 in advance. In this case, the amount of calculation by the control unit 50 can be reduced.

In the first and second embodiments, the display unit 43 is not limited to the display provided in the driver's

cab

60 or 60A, and any configuration can be adopted. For example, the display unit 43 may be a display of a portable terminal that can communicate with the

crane devices

1 and 1A directly or indirectly through a network or the like. When the

crane apparatus

1 or 1A is operated by the automatic operation by the control unit 50 without the operator (driver) manually driving, the display unit 43 may be omitted. That is, the

crane devices

1 and 1A may not include the display unit 43 when the control unit 50 performs the automatic operation instead of the manual operation of the worker.

The

crane devices

1 and 1A are not limited to the overhead crane and the container handling crane, respectively, and may be other various lanes.

1, 1A: crane device, 30, 30A: lifting tool, 40: imaging unit, 41: distance information acquisition unit, 42: color information acquisition unit, 43: display unit, 50: control unit, C: coil (load, target Object), D: Container (load), G: Locked part (object), M: Loading mark (object).

Claims

A lifting device for holding, holding and unloading a load;
An imaging unit which is provided on the hanger and which images a lower part of the hanger to obtain a captured image;
A distance information acquisition unit which is provided to the lifting device and obtains distance information to a plurality of measurement points within the range of the captured image below the lifting device;
A color information acquisition unit which is provided on the hanger and acquires color information within the range of the captured image of the lower side of the hanger;
A display unit for displaying the captured image;
A control unit configured to process the captured image and display the processed image on the display unit;
The control unit
Based on the distance information and the color information, a target area including an area where the target is located in the captured image is extracted from the captured image,
Detecting a corresponding portion corresponding to the shape of the reference image stored in advance in the target area;
The crane apparatus which makes the said display part display the said captured image emphasizing the said corresponding part detected.
The control unit
The crane apparatus according to claim 1, wherein the corresponding portion is not detected outside the target area of the captured image.
The control unit
Estimating a first region in which the target is located in the captured image based on the distance information;
Estimating a second region in which the target is located in the captured image based on the color information;
The crane apparatus according to claim 1, wherein a region including a region included in at least one of the first region and the second region is extracted from the captured image as the target region.
The control unit
Estimating a first region in which the target is located in the captured image based on the distance information;
Estimating a second region in which the target is located in the captured image based on the color information;
The crane apparatus according to claim 1, wherein an area including an area included in the first area and included in the second area is extracted from the captured image as the target area.
It has a moving mechanism for moving the lifting device,
The control unit sets a positional deviation between the corresponding portion in the captured image and a reference portion at which the corresponding portion should be positioned in the captured image when the lifting tool grips or unloads the load. Detect
The crane apparatus according to any one of claims 1 to 4, wherein the moving mechanism moves the lifting gear such that the positional deviation is eliminated.
The crane apparatus according to any one of claims 1 to 5, wherein the object is at least a part of the load gripped by the lifting gear.
The crane apparatus according to any one of claims 1 to 5, wherein the object is at least a part of a mounting portion on which the load held by the lifting tool is unloaded.