WO2017141320A1 - Dispositif de détermination, système de détermination, programme, et support d'enregistrement pour prendre en charge le fonctionnement de grue - Google Patents

Dispositif de détermination, système de détermination, programme, et support d'enregistrement pour prendre en charge le fonctionnement de grue Download PDF

Info

Publication number
WO2017141320A1
WO2017141320A1 PCT/JP2016/054304 JP2016054304W WO2017141320A1 WO 2017141320 A1 WO2017141320 A1 WO 2017141320A1 JP 2016054304 W JP2016054304 W JP 2016054304W WO 2017141320 A1 WO2017141320 A1 WO 2017141320A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
suspended load
determination
crane
recognized
Prior art date
Application number
PCT/JP2016/054304
Other languages
English (en)
Japanese (ja)
Inventor
基継 渡瀬
文彦 柴田
喬太 貞松
隆幸 河内
恵良 田畑
Original Assignee
株式会社大島造船所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社大島造船所 filed Critical 株式会社大島造船所
Priority to PCT/JP2016/054304 priority Critical patent/WO2017141320A1/fr
Priority to JP2017502908A priority patent/JP6266836B1/ja
Publication of WO2017141320A1 publication Critical patent/WO2017141320A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
    • B66C13/00—Other constructional features or details
    • B66C13/18—Control systems or devices
    • B66C13/46—Position indicators for suspended loads or for crane elements
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66C—CRANES; LOAD-ENGAGING ELEMENTS OR DEVICES FOR CRANES, CAPSTANS, WINCHES, OR TACKLES
    • B66C13/00—Other constructional features or details

Definitions

  • the present invention relates to a determination device for supporting crane operation.
  • Patent Document 1 A technique for reducing the burden on the crane operator described above has been proposed.
  • an operator's helmet is recognized from an image of a crane peripheral area captured from above to below a suspended load by a camera, and the recognized helmet image is displayed on the crane peripheral area image.
  • a safety confirmation device configured as described above has been proposed.
  • the safety confirmation device described in Patent Literature 1 sets a dangerous area centered on a suspended load position calculated based on the length, elevation angle, and direction of a jib (boom), and is a worker's helmet recognized in the dangerous area. Is displayed in a different color from other helmets, etc., to notify the crane operator that there is an operator in the danger area.
  • the safety confirmation device described in Patent Literature 1 changes the size of the dangerous area according to the suspended load amount identified from the wire tension and the suspended load height identified from the wire feed length or the like.
  • the safety confirmation device described in Patent Document 1 corrects the position of the dangerous area based on the wind direction and the wind speed measured by the sensor.
  • the safety confirmation device described in Patent Literature 1 expands the danger area in the jib movement direction in accordance with the turning of the jib.
  • the operator of the crane can easily know the worker in the danger area, and can perform an operation for avoiding danger without paying close attention. it can.
  • the safety confirmation device described in Patent Document 1 uses many types of information such as jib length, elevation angle, bearing, wire tension, wire feed length, wind direction, wind speed, and jib turning information. Is done. Therefore, in order to realize the safety confirmation device described in Patent Document 1, it is necessary to introduce many sensors.
  • the present invention aims to realize means for reducing the burden on the crane operator without using many sensors.
  • the present invention has a predetermined appearance feature in an image captured by an imaging device that is installed on a crane and performs imaging from above to below the suspended load of the crane.
  • Image recognition means for recognizing an object and the suspended load; time of image capture for each of images captured at a plurality of different times by the imaging device; and the object recognized by the image recognition means Based on the position of the object and the suspended load in the image, it is determined whether or not the object and the suspended load are closer than a predetermined distance in a period until a predetermined time elapses from the present time.
  • a determination device including determination means is proposed as a first aspect.
  • the determination unit recognizes the image capturing time and the image recognizing unit regarding each of images captured at a plurality of different times by the imaging device. Based on the position of the object and the suspended load in the image, the position of the object and the suspended load at a plurality of times within a period from the current time until the predetermined time elapses is estimated. And the structure of performing the said determination based on the said estimated position is proposed as a 2nd aspect.
  • the determination unit relates to each of images captured at a plurality of different times by the imaging device, and the image recognition time. Based on the position of the suspended load recognized by the means in the image, an approximate curve of the past movement path of the suspended load is specified, and the predetermined time from the present time is determined based on the specified approximate curve.
  • the structure of estimating the movement route of the suspended load in the period until it elapses is proposed as a third aspect.
  • the crane is disposed in a ship, and the determination unit is configured to move the suspended load recognized by the image recognition unit.
  • a configuration in which a vibration component included in a locus is specified and the determination is performed using the specified vibration component is proposed as a fourth aspect.
  • the determination unit is predetermined according to a moving speed in the image of the suspended load recognized by the image recognition unit.
  • a configuration in which the distance is changed is proposed as a fifth aspect.
  • the determination unit is configured to capture each image captured at a plurality of different times by the imaging device. Based on the position of the feature point of the suspended load recognized by the image recognition means, the state of rotation of the suspended load with the rope hanging the suspended load as an axis is identified, and the identified state of rotation is A configuration in which the above determination is performed is proposed as a sixth aspect.
  • the crane is disposed on a ship, and the image recognition unit has a predetermined appearance in an image captured by the imaging device.
  • a plurality of portions of the ship structure having the characteristics: and the determination means is recognized by the image recognition means in one of images taken at a plurality of different times by the imaging device.
  • Each of the plurality of portions recognized by the image recognition means in another position different from the one image among the positions of the plurality of portions and images picked up at a plurality of different times by the imaging device.
  • the relationship between the position of the object and the position of the suspended load recognized by the image recognition means in the one image and the position of the suspended load recognized by the image recognition means in the other image The configuration of specifying is proposed as a seventh aspect.
  • the display instructs the display device to display an image obtained by adding a graphic representing the determination result to the image captured by the imaging device.
  • the configuration of instruction means is proposed as an eighth aspect.
  • the present invention provides an image pickup device that is installed in a crane and picks up an image from above the crane's suspended load, and image data that is connected to the image pickup device and represents an image picked up by the image pickup device.
  • a determination system including the determination device according to any one of the first to eighth aspects described above that is received from the imaging device is proposed as a ninth aspect.
  • the present invention provides a computer having an object having a predetermined appearance characteristic in an image captured by an imaging device that is installed on a crane and performs imaging from above to below the suspended load of the crane.
  • the process of recognizing a suspended load, the time of capturing the image for each of images captured at a plurality of different times by the imaging device, the object recognized in the recognition process, and the image of the suspended load A program for executing a process of determining whether or not the object and the suspended load are closer than a predetermined distance in a period until a predetermined time elapses from the current time Is proposed as a tenth aspect.
  • the present invention provides a computer having an object having a predetermined appearance characteristic in an image captured by an imaging device that is installed on a crane and performs imaging from above to below the suspended load of the crane.
  • the process of recognizing a suspended load, the time of capturing the image for each of images captured at a plurality of different times by the imaging device, the object recognized in the recognition process, and the image of the suspended load A program for executing a process of determining whether or not the object and the suspended load are closer than a predetermined distance in a period until a predetermined time elapses from the current time Is proposed as an eleventh aspect of the present invention.
  • the determination apparatus when the current operation is continued using an image picked up from above the suspended load It is determined whether the suspended load and the object are in contact with or close to each other. As a result, the crane operator can easily ensure the safety of the jib movement.
  • the determination apparatus since it is determined whether or not the suspended load and the object are in contact with or close to each other based on the position of the suspended load and the object at each of a plurality of times. For example, even if it is estimated that the object and the suspended load pass through the same position with a time difference, an appropriate determination is made.
  • the shaking of the ship on which the crane is installed is considered.
  • the operator of the crane can ensure safety in moving the jib without necessarily paying close attention to the state of the ship's shaking.
  • the moving speed of the suspended load is taken into consideration in determining whether the suspended load and the object are in contact with or in proximity to each other.
  • the crane operator can operate the jib movement at high speed while ensuring safety.
  • the state of rotation of the suspended load is taken into account in determining whether the suspended load and the object are in contact with or in proximity to each other.
  • the operator of the crane can ensure safety in moving the jib without necessarily paying close attention to the state of rotation of the suspended load.
  • the elements associated with the change in the imaging position and the imaging direction included in the change in the position of the suspended load or the object in the image are removed. Even if it moves and the imaging position changes greatly, it is properly determined whether the suspended load and the object are in contact with or close to each other.
  • the operator of the crane can know whether there is a risk that the suspended load and the object are in contact with or close to each other by looking at the image displayed on the display device. it can.
  • the determination apparatus is realized by a computer.
  • the figure which illustrated the data structure of the hanging load position table memorize
  • FIG. 1 is a diagram schematically showing a determination system 1 and a ship 9 on which the determination system 1 is mounted.
  • the ship 9 is a bulk carrier and includes a plurality of cargo holds 91 and cranes 92 arranged according to the plurality of cargo holds 91.
  • the cargo hold 91 includes a main body 911 that forms a wall surface and a bottom surface, a hatch combing 912 provided around a hatch that is an opening above the main body 911, and a hatch cover 913 disposed on the hatch combing 912.
  • the cargo hold 91 illustrated in FIG. 1 includes two hatch covers 913 arranged on the bow side and the stern side, and each hatch cover 913 slides on the hatch coaming 912 and is folded to cover the hatch.
  • the hatch cover 913 of the cargo hold 91 illustrated on the right side of FIG. 1 is in an open state
  • the hatch cover 913 of the cargo hold 91 illustrated on the left side is in a closed state.
  • the crane 92 includes a crane post 921, a jib 922 extending outward from the crane post 921, a rope 923 hanging downward from the vicinity of the tip of the jib 922, and a grab 924 attached to the tip of the rope 923.
  • the suspended load of the crane 92 is a grab 924 and a cargo gripped by the grab 924.
  • An operation room 925 that is a space where an operator operates the crane 92 is provided on the upper part of the crane post 921.
  • the operator operates an operation lever or the like disposed in the operation chamber 925 while visually checking the position and shape of cargo, workers, heavy machinery, etc. in the cargo hold 91 and the position of the grab 924 from the operation chamber 925.
  • Various operations such as turning the jib 922, raising and lowering, expanding and contracting, feeding and winding the rope 923, and opening and closing the grab 924 are performed.
  • the determination system 1 is arranged according to each of the plurality of cranes 92.
  • the determination system 1 is arranged in the vicinity of the tip of the jib 922 and is arranged in the operation room 925, and is arranged in the operation room 925, and is directed to the operator such as workers and heavy equipment in the cargo hold 91.
  • a determination device 12 having a function of notifying the presence or absence of a danger of contact or proximity to the grab 924 is provided.
  • the imaging device 11 is attached to the jib 922 via an attachment device that suspends the imaging device 11 so that the imaging direction (optical axis) is always vertically downward, for example, due to gravity.
  • the imaging device 11 sequentially transmits image data representing an image captured every elapse of a predetermined time to the determination device 12.
  • the imaging device 11 and the determination device 12 are connected by communication.
  • the communication connection between the imaging device 11 and the determination device 12 may be performed via a communication cable such as an optical fiber, or may be performed wirelessly.
  • the determination device 12 is realized by a general computer performing processing according to a program.
  • the determination device 12 may be configured as a dedicated device.
  • FIG. 2 is a diagram illustrating a basic configuration of the computer 10 used as hardware of the determination device 12.
  • the computer 10 includes a memory 101 that stores various data, a processor 102 that performs various data processing in accordance with a program stored in the memory 101, a communication IF 103 that is an IF (Interface) that performs data communication with the imaging device 11, a user
  • a display device 104 such as a liquid crystal display for displaying an image
  • an operation device 105 such as a touch panel for receiving a user operation.
  • an external display device connected to the computer 10 may be used instead of the display device 104 built in the computer 10. Further, an external operation device connected to the computer 10 may be used instead of the operation device 105 built in the computer 10.
  • FIG. 3 is a diagram illustrating a functional configuration of the determination device 12.
  • the determination device 12 When the computer 10 performs processing according to the program for the determination device 12, the determination device 12 having the functional configuration shown in FIG. 3 is realized.
  • the functional configuration provided in the determination device 12 will be described below.
  • the determination device 12 includes a storage unit 121 that stores various data, an image data reception unit 122 that receives image data from the imaging device 11, and an image represented by the image data received by the image data reception unit 122 from the imaging device 11. Recognized by the image recognition means 123 and the image recognition means 123 for recognizing the grab 924, the objects such as workers and heavy machinery in the cargo hold 91, and the characteristic portions of the structure constituting the ship 9 by known image recognition processing. When the current movement state of the grab 924 and the object is maintained, the determination means 124 for determining whether or not they are in contact with or close to each other on a plane in the near future, the result of the determination by the determination means 124, etc. Display instruction means 125 is provided for instructing the display device 104 to display an image.
  • the storage means 121 stores in advance the suspended load feature data indicating the appearance characteristics of the grab 924, the object feature data indicating the appearance characteristics of the object regarding each of the objects such as workers and heavy machinery, and the ship 9 Structure feature data indicating the feature on the appearance of each of the plurality of portions constituting the portion is stored. These data are data used by the image recognition unit 123 to recognize the grab 924 or the like in the image captured by the imaging device 11.
  • FIG. 4 is a diagram illustrating the data structure of an image table that is a table used by the storage unit 121 to store image data.
  • the image data received by the image data receiving unit 122 from the imaging device 11 is accompanied by time data indicating the time at which the image represented by the image data was captured.
  • the image table has a “time” field for storing time data and an “image” field for storing image data.
  • the storage unit 121 stores data indicating the position in the image, such as a suspended load recognized by the image recognition unit 123.
  • FIG. 5 shows a suspended load position table which is a table used by the storage unit 121 to store data indicating the position of the grab 924 (suspended load), and the storage unit 121 is a position of an object such as an operator or a heavy machine. It is the figure which illustrated the data structure of the target object position table which is a table used in order to accumulate
  • the object position table is generated according to each object recognized by the image recognition unit 123.
  • the suspended load position table and the object position table store a “time” field that stores time data indicating the imaging time of an image used for recognition, and position data that indicates the position of the recognized suspended load or object. It has a “position” field and a “size” field for storing size data indicating the size of the recognized suspended load or object.
  • the size data stored in the “size” field indicates, for example, the diameter of a circle circumscribing the recognized suspended load or the like in the image used for recognizing the suspended load or the like.
  • the position data stored in the “position” field indicates, for example, the coordinates of the center position of a circle circumscribing the recognized suspended load or the like in the image used for recognizing the suspended load or the like.
  • FIG. 6 is a diagram illustrating a data structure of a structure position table, which is a table used by the storage unit 121 to store data indicating the position of the feature portion.
  • the structure position table is generated according to each of a plurality of preselected feature portions.
  • the structure position table has a “time” field for storing time data indicating the imaging time of an image used for recognition, and a “position” field for storing position data indicating the position of the recognized feature portion.
  • the position data stored in the “position” field indicates, for example, the coordinates of the recognized feature portion in the image used for recognition of the feature portion.
  • the image data receiving unit 122 receives the image data sequentially transmitted from the imaging device 11.
  • the image data received by the image data receiving means 122 is stored in the image table (FIG. 4).
  • the image recognition unit 123 uses the suspended feature data to display the grab 924 in the image represented by the image data received from the imaging device 11 by the image data reception unit 122 by a known image recognition process. Using the data, an object such as an operator or a heavy machine in the cargo hold 91 is recognized, and a feature portion (two corners of the hatch combing 912) is recognized using the structure feature data. Data indicating the position of the grab 924 and the like recognized by the image recognition means 123 is stored in the suspended load position table (FIG. 5), the object position table (FIG. 5), or the structure position table (FIG. 6). .
  • the determination unit 124 determines whether or not they will contact or approach on a plane in the near future. judge. Note that “contact on a plane” means that the grab 924 passes just above the object even if it is not in contact in the three-dimensional space.
  • FIG. 7A and FIG. 7B are diagrams illustrating a flow of processing performed by the determination unit 124 every time a predetermined time elapses.
  • the determination means 124 reads the record with the second most recent time indicated by the time data (hereinafter referred to as the record at time (k-1)) from the suspended load position table (FIG. 5), and sets the time (k-1)
  • the coordinates of the grab 924 indicated by the position data of the record are the latest image (the image captured at time (k)).
  • a process of converting to coordinates in the coordinate system is performed (step S101).
  • the coordinate conversion in the process of step S101 is performed by converting the coordinates of a plurality of feature portions (two corner portions of hatch coaming 912) in the image captured at time (k-1) in the image captured at time (k). This is done by mapping corresponding to the coordinates of these feature portions.
  • the determination unit 124 reads the records of time (k) and time (k ⁇ 1) from the structure position table (FIG. 6) corresponding to each of the plurality of characteristic portions, and the positions of the read records.
  • the coordinates indicated by the position data of the record at time (k ⁇ 1) read from the suspended load position table are converted by a known matrix calculation using the coordinates indicated by the data.
  • the determination means 124 reads the record at time (t) from the suspended load position table (FIG. 5), and starts from the coordinates (coordinate of grab 924 at time (k-1)) obtained by the conversion in step S101. Then, a vector whose end point is the coordinate indicated by the position data of the record at the read time (t) (the coordinate of the grab 924 at time (k)) is specified (step S102). This vector indicates the moving direction and moving speed of the grab 924 at the present time. The determination unit 124 temporarily stores the vector data indicating the vector specified in step S102 in the storage unit 121.
  • the determination unit 124 assigns an initial value “1” to the counter i used for sequentially selecting the objects recognized by the image recognition unit 123 (step S103).
  • the number of objects recognized by the image recognition unit 123 is n.
  • the determination unit 124 performs the same processing as steps S101 and S102 on the i-th object among the objects recognized by the image recognition unit 123 (steps S104 and S105). Thereby, a vector indicating the moving direction and moving speed of the i-th object is specified.
  • the determination unit 124 temporarily stores the vector data indicating the vector specified in step S105 in the storage unit 121.
  • the determination unit 124 subsequently sets a distance threshold D corresponding to the moving speed of the grab 924 indicated by the vector identified in step S102.
  • the threshold value D means a distance that can be considered that there is no risk that the grab 924 and the object are in contact with each other if the distance between the grab 924 and the object is more than that distance.
  • the determination unit 124 sets a larger threshold value D as the moving speed of the grab 924 increases. For example, the determination unit 124 calculates the threshold value D as a predetermined function (a monotonically increasing function of the moving speed of the grab 924) having the moving speed of the grab 924 as a variable.
  • the determination unit 124 assigns an initial value “0” to a variable t indicating the elapsed time from the current time, and a flag f (1) indicating whether or not an alarm is required for contact or proximity between the grab 924 and the object. Substitute the initial value “0” for f (n) (step S109).
  • the flag f (i) is a flag corresponding to the i-th object.
  • the determination unit 124 adds a predetermined time ⁇ t to the variable t (step S110), and determines whether or not the variable t is larger than a predetermined time threshold T (step S111).
  • the threshold value T is assumed that when the current moving state of the grab 924 and the target object (in this embodiment, the moving direction and moving speed) is maintained, they contact each other on the plane after the time has elapsed from the present time. This also means that the contact can surely be avoided if the operator of the crane 92 performs an appropriate operation for changing the moving state of the grab 924 before the passage of the time.
  • step S111 When the variable t is equal to or smaller than the threshold T (step S111; “No”), the determination unit 124 assumes that the moving direction and the moving speed indicated by the vector specified in step S102 are maintained, and the time t has elapsed from the present time.
  • the position of the grab 924 at that time (the coordinates in the coordinate system of the image captured at time (k)) is estimated (step S112).
  • the determination unit 124 substitutes the initial value “1” for the counter i (step S113). Subsequently, regarding the i-th object, the determination unit 124 assumes that the same processing as in step S112, that is, the moving direction and the moving speed indicated by the vector specified in step S105 are maintained, and the time t from the present time is maintained. The position of the target object at the point of time (the coordinates in the coordinate system of the image captured at time (k)) is estimated (step S114).
  • the determination device 12 determines whether the distance d between the position of the grab 924 estimated in step S112 and the position of the first object estimated in step S114 is less than the threshold D set in step S108. Is determined (step S115).
  • the determination unit 124 sets the flag f ( “1” indicating that a warning is required is substituted for i) (step S116).
  • step S115 when the distance d between the position of the grab 924 and the position of the first object when the time t has elapsed from the current time is greater than or equal to the threshold value D (step S115; “No”), the determination unit 124 sets the flag No change is made to f (i).
  • step S111 If the variable t exceeds the threshold T in the determination in step S111 (step S111; “Yes”), the determination unit 124 temporarily stores the flags f (1) to f (n) at that time in the storage unit 121. To end the series of processing.
  • FIG. 8 is a diagram illustrating an image displayed on the display device 104 in accordance with an instruction from the display instruction unit 125.
  • the image in FIG. 8 is obtained by cutting out the area surrounded by the hatch coaming 912 from the latest image captured by the imaging device 11, the circle circumscribing each of the grab 924 and the target object, their moving direction, and It is the image which added the arrow which shows a moving speed.
  • the display instruction means 125 performs the following processing every time a predetermined time elapses, and generates an image that instructs the display device 104 to display. First, the display instruction unit 125 reads the following data.
  • the display instruction means 125 is centered on the position indicated by the position data read in (b) above the image represented by the image data read in (a), and the size read in (b) above. A circle with the diameter indicated by the data is added. Further, the display instruction unit 125 adds an arrow having a direction and a length corresponding to the vector indicated by the vector data read in (d) so as to extend outward from the added circle. At this time, if all of the flags f (1) to f (n) read in (f) above indicate “0”, the display instruction means 125 adds a blue circle and an arrow, for example, When “1” is indicated, the display instruction unit 125 adds, for example, a red circle and an arrow.
  • the display instruction unit 125 adds a circle having a diameter centered on the position indicated by the position data read in (c) above and the diameter indicated by the size data read in (c) above. Then, an arrow having a direction and length corresponding to the vector indicated by the vector data read in (e) is added so as to extend outward from the added circle.
  • the display instruction means 125 is, for example, blue
  • the display instruction unit 125 adds, for example, a red circle and an arrow.
  • the display instructing unit 125 includes two positions indicated by the position data read out in (g) above (the corner portion on the bow side and starboard side of the hatch coaming 912 and the image including the circle and the arrow as described above). Cut out a rectangular area whose diagonal is the line connecting the stern and port side corners).
  • the display instruction unit 125 enlarges or reduces the cut-out rectangular area image so as to have a predetermined size. At that time, the display instruction means 125 adjusts the thickness of the circle or arrow line added to the image to be the same regardless of the enlargement or reduction ratio.
  • the display instruction unit 125 outputs the image generated in this way to the display device 104 and instructs the display of the image. As a result, an image as illustrated in FIG. 8 is displayed on the display device 104.
  • the operator When the operator operates the crane 92, the operator sees the image (FIG. 8) displayed on the display device 104, and moves the grab 924 in the moving direction and moving speed, as well as the cargo hold 91 such as an operator and heavy equipment.
  • the movement direction and the movement speed of each of the objects can be intuitively known, and whether or not the grab 924 may contact or approach each of the objects can be intuitively known.
  • the operator can freely operate the crane 92 while no red circle or arrow is displayed on the display device 104, so that work efficiency is improved. If a red circle or arrow is displayed on the display device 104, the contact speed or the proximity of the grab 924 and the object is avoided by reducing the moving speed of the jib 922 or changing the moving direction. be able to.
  • the burden on the operator of the crane 92 is reduced, and the work efficiency is improved and the safety is ensured.
  • the determination means 124 is a period during which the threshold T of time elapses from the present time for the moving direction and moving speed of the suspended load and the object at the present time (hereinafter referred to as “future period” for convenience). ), Estimating future positions of suspended loads and objects, assuming they are maintained.
  • the method by which the determination unit 124 estimates the future position of the suspended load and the object is not limited to this.
  • the determination unit 124 specifies the acceleration in movement in addition to the moving direction and moving speed based on the temporal change of the past position of the suspended load and the object, and the specified acceleration is maintained in the future period. Assuming that, the future position of suspended loads and objects may be estimated.
  • the determination unit 124 specifies an approximate curve of the movement path indicated by a change in the past position of the suspended load as information indicating the movement state of the suspended load by a known method, and the suspended load is based on the identified approximate curve.
  • the movement path in the future period may be estimated.
  • FIG. 9 is a diagram for explaining processing performed by the determination unit 124 in this modification.
  • a solid curve AB shown in FIG. 9 indicates a path along which the grab 924 has moved in the past period up to the present time. Note that point B is the current position of the grab 924.
  • the determination unit 124 identifies an approximate curve of the curve AB.
  • a dashed curve AC shown in FIG. 9 shows an example of an approximate curve specified by the determination means 124.
  • the movement path of the grab 924 is a component of movement along a circle centering on the axis of the crane post 921 accompanying the turning of the jib 922, and a circle centering on the axis of the crane post 921 accompanying undulation or expansion / contraction of the jib 922. It is determined by the combination of the component of movement along the radial direction of. Therefore, it is desirable to use a non-linear curve such as a polynomial curve as an approximate curve of the moving route of the crane's suspended load.
  • the determination means 124 estimates the future position of the suspended load when the suspended load moves along the specified approximate curve in the future period, and uses the estimated position to make contact between the suspended load and the object or Make a proximity decision.
  • the determination unit 124 specifies the vibration component included in the movement trajectory in the image of the suspended load as information indicating the movement state of the suspended load, and uses the identified vibration component to contact the suspended load with the object or You may make a proximity determination.
  • the solid curve AB shown in FIG. 9 was obtained by interpolating the coordinates of the grab 924 in each coordinate system of a plurality of images captured in the past into coordinates in the coordinate system of the latest image. It is a curve.
  • FIG. 10 illustrates a plurality of images captured in the past, starting from a coordinate obtained by converting the coordinates of the center point of each image in the coordinate system of the plurality of images captured in the past into coordinates in the coordinate system of the latest image. It is the figure which showed the time-dependent change of the vector which made the end point what converted the coordinate of the grab 924 in each coordinate system into the coordinate in the coordinate system of the newest image.
  • the directions of the vectors shown in FIG. 10 are approximately 1:30 and 7:30, and the lengths of these vectors periodically increase and decrease.
  • the determination unit 124 identifies a vector that changes with time as illustrated in FIG. 10, and represents the vibration direction of the grab 924 by specifying the direction of the identified vector and the period and amplitude of increase / decrease of the vector length. Identify as information.
  • the determination means 124 estimates the future position of the suspended load, assuming that the suspended load will continue to vibrate in the direction, period and amplitude so identified during the future period, and using the position thus estimated, Judgment of contact or proximity between suspended load and object.
  • a dashed curve AC shown in FIG. 11 is a curve obtained by adding the vibration component specified as described above to the dashed approximate curve AC shown in FIG.
  • the determination means 124 estimates the future position of the suspended load when the suspended load moves along the dashed curve AC shown in FIG. 11, and uses the estimated position to determine the suspended load and the target. Judgment of contact or proximity to objects.
  • the determination means 124 specifies the rate of change (angular velocity) of the vibration direction in addition to the current vibration direction, period, and amplitude as information representing the vibration component included in the trajectory of the suspended load. It may be used to determine contact or proximity between the object and the object.
  • the determination unit 124 uses information indicating the state of rotation of the suspended load about the rope 923 as information indicating the moving state of the suspended load.
  • the information indicating the rotation state of the specified suspended load may be specified, and the contact or proximity of the suspended load and the object may be determined.
  • FIG. 12 is a diagram illustrating the data structure of the suspended load position table used in this modification.
  • the position of the suspended load recognized by the image recognition means 123 in addition to the center position, the position of the connection point between the rope 923 and the suspended load (connection point position) and the positions of the four corners of the suspended load (the first position) 1 corner position to 4th corner position) are specified.
  • FIG. 13 shows the coordinates of the connecting point position and the coordinates of the first corner position stored in the suspended load position table (FIG. 12) converted to the coordinates in the coordinate system of the latest image, and the converted coordinates of the connecting point position.
  • FIG. 6 is a diagram showing a change over time of a vector having a starting point as a starting point and the coordinates of the converted first corner position as an end point. The direction of the vector shown in FIG. 13 changes periodically between the 11 o'clock direction and the 1 o'clock direction.
  • the determination unit 124 identifies a vector that changes with time as illustrated in FIG. 13, and the direction of change and the speed of change (angular velocity) of the specified vector are defined as a suspended load with the rope 923 as an axis. It is specified as information indicating the state of rotation. Note that, as illustrated in FIG. 13, when the direction of the vector periodically changes within a certain angle range, the determination unit 124 specifies the angle range and the period as information indicating the state of rotation. Further, when the vector direction changes in the same direction at a substantially constant angular velocity, the determination unit 124 specifies the rotation direction and velocity (angular velocity) as information indicating the rotation state.
  • FIG. 14 is a diagram illustrating a state in which the suspended load moves with rotation about the rope 923 as an example in which the suspended load is not a glove 924 but a long member.
  • the rectangle indicated by the solid line indicates the past position (posture) of the suspended load recognized by the image recognition means 123
  • the curve indicated by the broken line indicates an approximate curve of the moving path at the center of the suspended load.
  • the rectangle indicated by the broken line in FIG. 14 indicates that the center of the suspended load moves along the approximate curve in the future period, and the rotation state specified as described above is maintained in the future period. Shows the future position (posture) of the suspended load.
  • the determination means 124 estimates the position (posture) of the suspended load indicated by the broken-line rectangle in FIG. 14, and determines the contact or proximity between the suspended load and the object based on the estimated position (posture) of the suspended load. Do.
  • the determination means 124 may specify the height of the suspended load and use the identified suspended load height to determine contact or proximity between the suspended load and the object.
  • the higher the suspended load the larger the size of the suspended load relative to the size of the structure of the ship 9 in the image.
  • the size of the suspended load relative to the distance between the two feature portions of the structure of the ship 9 recognized by the image recognition means 123 in the image by the image recognition means 123 for example, the two corner portions of the hatch combing 912).
  • a configuration may be employed in which the distance threshold D is set to be larger as the ratio is larger.
  • the determination device 12 may include a sound generation instruction unit that instructs the sound generation device to generate a sound according to the determination result by the determination unit 124.
  • the determination device 12 may include a control instruction unit that instructs the control device of the crane 92 to execute control according to the determination result by the determination unit 124. For example, when the determination unit 124 sets a distance threshold E having a value smaller than the threshold D in addition to the distance threshold D used in the above-described embodiment, the suspended load and the object are closer to each other than the threshold E distance. If estimated, a configuration in which the control instruction means instructs the control device of the crane 92 to control the speed of movement of the jib 922 may be employed.
  • the suspended load is a grab, but the form and type of the suspended load are not limited.
  • the use of the determination system according to the present invention is not limited to a crane mounted on a ship.
  • the determination system according to the present invention may be employed in a crane used on the ground.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Image Analysis (AREA)
  • Control And Safety Of Cranes (AREA)

Abstract

La présente invention vise à fournir un moyen pour soulager la charge sur un opérateur de grue, sans utiliser un grand nombre de capteurs. À cet effet, la présente invention concerne un système de détermination (1) selon un mode de réalisation. Le système de détermination (1) comprend : un dispositif d'imagerie (11) qui est positionné près de la pointe de la flèche (922) d'une grue (92) et qui capture des images de la zone se trouvant au-dessous de manière continue ; et un dispositif de détermination (12) qui reconnaît des images capturées par l'unité d'imagerie (11), détermine l'état de mouvement du grappin (924) et d'objets tels que des ouvriers, des équipements lourds et analogues à l'intérieur de l'entrepôt de fret (91) dans les images et, sur la base de l'état de mouvement qui a été déterminé, détermine s'il existe une possibilité de contact ou de rapprochement entre le grappin (924) et un objet dans une vue plane dans le futur proche. S'il est estimé qu'il existe une possibilité de contact ou de rapprochement entre le grappin (924) et l'objet, le dispositif de détermination (12) informe l'opérateur de la grue (92).
PCT/JP2016/054304 2016-02-15 2016-02-15 Dispositif de détermination, système de détermination, programme, et support d'enregistrement pour prendre en charge le fonctionnement de grue WO2017141320A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2016/054304 WO2017141320A1 (fr) 2016-02-15 2016-02-15 Dispositif de détermination, système de détermination, programme, et support d'enregistrement pour prendre en charge le fonctionnement de grue
JP2017502908A JP6266836B1 (ja) 2016-02-15 2016-02-15 クレーンの操作を支援するための判定装置、判定システム、プログラムおよび記録媒体

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/054304 WO2017141320A1 (fr) 2016-02-15 2016-02-15 Dispositif de détermination, système de détermination, programme, et support d'enregistrement pour prendre en charge le fonctionnement de grue

Publications (1)

Publication Number Publication Date
WO2017141320A1 true WO2017141320A1 (fr) 2017-08-24

Family

ID=59625730

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/054304 WO2017141320A1 (fr) 2016-02-15 2016-02-15 Dispositif de détermination, système de détermination, programme, et support d'enregistrement pour prendre en charge le fonctionnement de grue

Country Status (2)

Country Link
JP (1) JP6266836B1 (fr)
WO (1) WO2017141320A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019172413A1 (fr) * 2018-03-09 2019-09-12 株式会社タダノ Grue
JP2020004329A (ja) * 2018-07-02 2020-01-09 日本製鉄株式会社 情報処理システム、情報処理装置、情報処理方法、プログラム、及び、記憶媒体
EP3558857A4 (fr) * 2016-12-20 2020-08-19 Konecranes Global OY Procédé, programme informatique et équipement de commande d'une grue et procédé de mise à jour d'une grue
WO2020166455A1 (fr) * 2019-02-14 2020-08-20 株式会社タダノ Grue et système de génération de chemin
WO2022075340A1 (fr) * 2020-10-09 2022-04-14 株式会社五合 Dispositif de traitement d'informations pour grue
WO2023162201A1 (fr) * 2022-02-28 2023-08-31 株式会社エムエムアイ Système de commande d'équipement de grue et procédé de commande d'équipement de grue

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4505558B2 (ja) * 2004-05-12 2010-07-21 独立行政法人海上技術安全研究所 船舶用航行支援装置
JP2010241548A (ja) * 2009-04-03 2010-10-28 Kansai Electric Power Co Inc:The クレーンの安全確認装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4505558B2 (ja) * 2004-05-12 2010-07-21 独立行政法人海上技術安全研究所 船舶用航行支援装置
JP2010241548A (ja) * 2009-04-03 2010-10-28 Kansai Electric Power Co Inc:The クレーンの安全確認装置

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3558857A4 (fr) * 2016-12-20 2020-08-19 Konecranes Global OY Procédé, programme informatique et équipement de commande d'une grue et procédé de mise à jour d'une grue
WO2019172413A1 (fr) * 2018-03-09 2019-09-12 株式会社タダノ Grue
JP2019156533A (ja) * 2018-03-09 2019-09-19 株式会社タダノ クレーン
JP2020004329A (ja) * 2018-07-02 2020-01-09 日本製鉄株式会社 情報処理システム、情報処理装置、情報処理方法、プログラム、及び、記憶媒体
JP7173176B2 (ja) 2019-02-14 2022-11-16 株式会社タダノ クレーン及び経路生成システム
JPWO2020166455A1 (ja) * 2019-02-14 2021-11-18 株式会社タダノ クレーン及び経路生成システム
WO2020166455A1 (fr) * 2019-02-14 2020-08-20 株式会社タダノ Grue et système de génération de chemin
WO2022075340A1 (fr) * 2020-10-09 2022-04-14 株式会社五合 Dispositif de traitement d'informations pour grue
JPWO2022075340A1 (fr) * 2020-10-09 2022-04-14
JP2022185104A (ja) * 2020-10-09 2022-12-13 株式会社五合 クレーン用情報処理装置
JP2022185103A (ja) * 2020-10-09 2022-12-13 株式会社五合 クレーン用情報処理装置
JP2022188244A (ja) * 2020-10-09 2022-12-20 株式会社五合 クレーン用情報処理装置
JP7228944B2 (ja) 2020-10-09 2023-02-27 株式会社五合 クレーン用情報処理装置
JP7258388B2 (ja) 2020-10-09 2023-04-17 株式会社五合 クレーン用情報処理装置
JP7258389B2 (ja) 2020-10-09 2023-04-17 株式会社五合 クレーン用情報処理装置
JP7289581B2 (ja) 2020-10-09 2023-06-12 株式会社五合 クレーン用情報処理装置
WO2023162201A1 (fr) * 2022-02-28 2023-08-31 株式会社エムエムアイ Système de commande d'équipement de grue et procédé de commande d'équipement de grue

Also Published As

Publication number Publication date
JP6266836B1 (ja) 2018-01-24
JPWO2017141320A1 (ja) 2018-02-22

Similar Documents

Publication Publication Date Title
JP6266836B1 (ja) クレーンの操作を支援するための判定装置、判定システム、プログラムおよび記録媒体
US10857675B2 (en) Control device, robot, and robot system
US9050722B2 (en) Pickup device capable of determining holding position and posture of robot based on selection condition
JP6481819B2 (ja) 移動式クレーンの表示装置及び移動式クレーンの同期装置。
JP6017691B2 (ja) 情報提示装置、クレーンシステム及び情報提示方法
JP6790932B2 (ja) 飛行体運用システム、クレーン装置制御方法、及び制御プログラム
JP2011079648A (ja) 固定映像表示システム
WO2011060640A1 (fr) Équipement de détection de pose de crochet et grue
CN110526142A (zh) 智能塔吊
JP2013142037A (ja) 作業車用の吊荷監視カメラ装置
CN109405804B (zh) 作业辅助方法及系统
JP2016153335A (ja) 吊荷監視装置
WO2017208435A1 (fr) Dispositif de traitement de données, procédé et programme d'identification d'une position de cargaison levée de grue
JP2016166086A (ja) 画像表示装置
JP2016097925A (ja) 操船支援システム及びそのシステムを搭載した船舶
JP6053141B2 (ja) 作業確認装置
EP3926976B1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
WO2018178506A1 (fr) Procédé, système et dispositif pour afficher des images vidéo en temps réel de l'environnement d'un véhicule
JP6162481B2 (ja) 作業領域線表示装置
CN109927058B (zh) 抓握装置、抓握判定方法和抓握判定程序
JP7107447B2 (ja) クレーン情報表示システム
JP2019218198A (ja) 操作支援システム
JP2018056908A (ja) 情報処理装置、情報処理方法およびプログラム
JP6368503B2 (ja) 障害物監視システム及びプログラム
JP6005496B2 (ja) 遠隔監視装置および遠隔監視方法

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2017502908

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16890463

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16890463

Country of ref document: EP

Kind code of ref document: A1