JP2021143038A - Operation supporting system and operation supporting method of crane - Google Patents

Abstract

To provide an operation supporting system and an operation supporting method of a crane capable of improving workability of crane operations by an operator.SOLUTION: An operation supporting system of a crane takes a photographed image P1 including a suspended object 30 by a photographing device 21 and acquires two-dimensional coordinates in the photographed image P1 of a subject reflected in the photographed image P1. The system generates a processed image P3 in which the visibility of a preselected subject is processed relatively higher than the visibility of other subjects unselected based on the acquired two-dimensional coordinates of the subject, where the preselected subject includes at least one of the suspended object 30, a landing target surface 31 on which the suspended object 30 is landed, and an obstacle 33 on the cargo handling path of the suspended object 30 of the subjects reflected in the image P1. The system causes a display device 23 to display the generated, processed image P3.SELECTED DRAWING: Figure 5

Description

The present invention relates to a crane operation support system and an operation support method, and more particularly to a crane operation support system and an operation support method capable of improving the workability of a crane operation by an operator.

As a method of supporting the landing operation of landing a suspension body (suspension device or a load gripped by the suspension device) suspended from the trolley of a container crane on the landing target surface (ground, load, etc.), the trolley is used. A crane operation method is proposed in which the suspended body and the landing target surface below it are photographed from an oblique direction with the installed imaging device (camera), and the captured image captured by the imaging device is displayed on the display device (display device). (See, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 5-246683

By the way, as in the operation method described in Patent Document 1, the photographed image taken by the photographing device installed in the trolley is arranged in the structure of the crane, the container terminal, and the ship in addition to the suspended body and the landing target surface. Various subjects such as containers and equipment, and transporters (also called chassis) that transport containers are captured. The captured image includes subjects that the operator needs to recognize when performing cargo handling work such as the suspension body, the landing target surface, and obstacles existing on the cargo handling path of the suspension body, and other subjects that need not be recognized. Since the image is mixed with the subject, the amount of information that the operator can see is very large. Therefore, it is difficult to grasp a specific subject that the operator needs to recognize when performing cargo handling work, among other subjects that need not be recognized, by simply displaying the captured image on the display device as it is. ..

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a crane operation support system and an operation support method capable of improving the workability of a crane operation by an operator.

The crane operation support system of the present invention for achieving the above object captures a photographed image including a suspended body suspended from a trolley by a photographing device, and is based on the photographed image taken by the photographing device. In the crane operation support method for displaying an image on a display device, the step of capturing the captured image including the suspended body by the imaging device and the two-dimensional coordinates of the subject reflected in the captured image in the captured image are acquired. Based on the step and the acquired two-dimensional coordinates, at least of the suspension body, the landing target surface on which the suspension body is landed, or an obstacle existing on the cargo handling path of the suspension body among the subjects. A step of generating a processed image in which the visibility of the preselected subject including any of the above is relatively higher than the visibility of the other unselected subject, and the generated processed image are described. It is characterized by including a step of displaying on the display device.

The crane operation support method of the present invention for achieving the above object is based on a photographed image including a suspended body suspended from a trolley by a photographing device and photographed by the photographing device. In the crane operation support method for displaying an image on a display device, the step of capturing the captured image including the suspended body by the imaging device and the two-dimensional coordinates of the subject reflected in the captured image in the captured image are acquired. Based on the step and the acquired two-dimensional coordinates, at least of the suspension body, the landing target surface on which the suspension body is landed, or an obstacle existing on the cargo handling path of the suspension body among the subjects. A step of generating a processed image in which the visibility of the preselected subject including any of the above is relatively higher than the visibility of the other unselected subject, and the generated processed image are described. It is characterized by including a step of displaying on the display device.

According to the present invention, the visibility of a preselected subject including at least one of a hanging body, a landing target surface, and an obstacle is selected among the subjects shown in the captured image captured by the photographing device. The display device displays a processed image that is higher than the visibility of other subjects. As a result, a specific subject (suspended body, landing target surface, or obstacle) that the operator operating the crane needs to recognize when performing cargo handling work, rather than displaying the captured image as it is on the display device. At least one) will be easier to understand. Therefore, the workability of the crane operation by the operator can be improved.

It is a block diagram which illustrates embodiment of the crane operation support system of this invention. It is a block diagram which illustrates the operation support system of FIG. It is explanatory drawing which illustrates the photographed image photographed by the photographing apparatus of FIG. It is explanatory drawing which illustrates the upper layer image generated by the processing apparatus of FIG. It is explanatory drawing which illustrates the processed image displayed on the display device of FIG. It is a flow figure which illustrates embodiment of the operation support method of this invention. It is explanatory drawing which illustrates another processed image displayed on the display device.

Hereinafter, embodiments of the crane operation support system and the operation support method of the present invention will be described. In the figure, the traveling direction of the trolley 5 of the crane 1 is the X direction, the direction orthogonal to the X direction in the horizontal direction is the Y direction, and the vertical direction (vertical direction) is the Z direction.

The operation support system 20 for the crane 1 of the embodiment illustrated in FIG. 1 is a system that supports the operation of the crane 1 by an operator. The operation support system 20 mainly performs a landing operation (hereinafter referred to as a landing operation) in which the suspension body 30 suspended from the trolley 5 is landed on the landing target surface 31 located below the suspension body 30. Support. The operation support system 20 can also be used for the operation of the crane 1 other than the landing operation.

The above-mentioned suspension body 30 is a single suspension tool 7 when the crane 1 is used to grab the load, and the landing target surface 31 in this case is the upper surface of the load to be handled. Is. The hanging body 30 when going to put the hanging load held by the hanging tool 7 is the hanging tool 7 and the hanging load, and the landing target surface 31 in this case is the loading placed on the land 36 or the ship 35. The upper surface of the land 36, the floor of the ship 35, the upper surface of the loading platform of the carrier 34, and the like.

In this embodiment, an operation support system 20 when mounted on a crane 1 that handles cargo handling of a container 32 for a ship 35 is illustrated. In the following, a landing operation is performed in which the container hanger 7 (also referred to as a spreader) provided in the crane 1 is landed on the upper surfaces of two adjacent containers 32 mounted on the loading platform of the transporter 34. Illustrate the case. That is, the suspension body 30 in this embodiment is the suspension tool 7, and the landing target surface 31 is the upper surface of the two containers 32.

As illustrated in FIG. 1, the crane 1 moves in the X direction supported by the leg structure 2, the girder portion 4 extending in the X direction supported on the upper portion of the leg structure 2, and the girder portion 4. The trolley 5 is provided with a hanging tool 7 suspended from the trolley 5 by a wire 6. The leg structure 2 includes a plurality of legs 2a extending in the Z direction, a horizontal girder 2b connecting the legs 2a adjacent to each other in the X direction, and a horizontal beam 2c (sill beam) connecting the legs 2a adjacent to each other in the Y direction. It is also composed of). At the lower end of each leg 2a, a traveling device 3 that can travel along a rail laid on land and extending in the Y direction is installed.

A machine room 8 is provided in the upper part of the leg structure 2, and a moving device 9 for moving the trolley 5 in the X direction in the machine room 8 and a hanger 7 for winding or unwinding the wire 6 in the Z direction. An elevating device 10 is installed. In the machine room 8, a crane control device 11 that controls a traveling device 3, a moving device 9, and an elevating device 10 is further installed. The installation positions of the moving device 9, the lifting device 10, and the crane control device 11 are not limited to the machine room 8, and may be arranged at other positions such as the trolley 5.

The operation chamber 12 of the crane 1 of this embodiment is arranged at a position separated from the crane 1. In the operation room 12, an operation device 13 that is communicably connected to the crane control device 11 via a wireless antenna is installed. The operator of the crane 1 operates the operating device 13 in the operation room 12 to remotely control the crane 1 (traveling device 3, moving device 9, and lifting device 10) via the crane control device 11. The operation chamber 12 may be attached to the structure of the crane 1.

As illustrated in FIG. 2, the operation support system 20 includes a photographing device 21, a processing device 22, a display device 23, and an input means 24. The processing device 22 has a setting unit 22a, a position information acquisition unit 22b, a generation unit 22c, and a display control unit 22d as functional elements. The processing device 22, the display device 23, and the input means 24 are arranged in the operation room 12. Each functional element constituting the processing device 22 can be arranged in a place other than the operation room 12. The photographing device 21, the display device 23, and the input means 24 are connected to the processing device 22 so as to be able to communicate with each other by wire or wirelessly.

The photographing device 21 is installed at a position where the hanging body 30 can be photographed, and sequentially photographs the photographed image P1 including the suspending body 30. Examples of the photographing device 21 include a digital video camera and the like. As illustrated in FIG. 1, the photographing apparatus 21 of this embodiment is installed at the land side end portion of the trolley 5 in the X direction in a side view (viewpoint viewed from the Y direction). The photographing direction of the photographing device 21 is set diagonally downward from the land side in the X direction to the sea side in the X direction.

By taking a bird's-eye view of the suspended body 30 from an oblique direction with the photographing device 21 installed in the trolley 5, as illustrated in FIG. 3, a conventional operator who operates a crane in the driver's cab attached to the trolley 5 The photographed image P1 that reproduces the field of view is photographed. The installation position and the photographing direction of the photographing device 21 are not particularly limited as long as the photographed image P1 including the suspension body 30 can be photographed. The photographing device 21 can be arranged, for example, in the leg structure 2, the girder portion 4, or the like, or can be arranged at a position away from the crane 1. The captured image P1 captured by the photographing device 21 is configured to be sequentially input to the processing device 22.

The processing device 22 acquires the two-dimensional coordinates of the photographed image P1 of the subject reflected in the photographed image P1. Then, based on the acquired two-dimensional coordinates of the subject, the processing device 22 implants the suspension body 30 and the suspension body 30 among the subjects reflected in the captured image P1 as illustrated in FIG. The visibility of a preselected subject including at least one of the target surface 31 or the obstacle 33 existing on the cargo handling path of the suspension body 30 is defined as that of another unselected subject (hereinafter referred to as another subject). A processed image P3 that is relatively high in terms of visibility is generated. The processing device 22 is configured to display the generated processed image P3 on the display device 23.

Among the subjects reflected in the captured image P1, the main subjects involved in the operation of the crane 1 by the operator are the suspension body 30, the landing target surface 31 which is the target for landing the suspension body 30, and the suspension body 30. It is an obstacle 33 on the cargo handling route of. Therefore, in the operation support system 20, at least one of the suspension body 30, the landing target surface 31, and the obstacle 33, which the operator needs to recognize by performing image processing on the captured image P1 by the processing device 22. A processed image P3 is generated that makes it easier to visually identify the image P1 than the captured image P1.

Specifically, the above-mentioned obstacle 33 is installed on a horizontal beam 2c or a leg structure 2 which is located within the moving range of the suspension body 30 and may come into contact with or collide with the suspension body 30. These include incidental equipment, structures and equipment of the ship 35, loads and equipment other than the landing target surface 31 mounted on the ship 35 and the land 36, and a person existing on the cargo handling route of the suspension body 30.

As an image process for increasing the visibility of a pre-selected subject relative to the visibility of other subjects, the outer edge of the selected subject is more emphasized with respect to the captured image P1. Image processing that performs processing (so-called sharpness), image processing that superimposes an additional marking 50 that emphasizes the position or shape of each selected subject on the captured image P1, and other image processing compared to the captured image P1. Examples include image processing that reduces the visibility of the subject.

The processing device 22 is hardware composed of a CPU that performs various information processing, an internal storage device that can read and write programs and information processing results used for performing the various information processing, and various interfaces. The setting unit 22a, the position information acquisition unit 22b, the generation unit 22c, and the display control unit 22d constituting the processing device 22 are stored as programs in the internal storage device, and are executed by the CPU in a timely manner. Each functional element constituting the processing device 22 may be a PLC that functions independently in addition to the program.

The setting unit 22a selects a subject whose visibility is relatively high. The setting unit 22a of this embodiment is further configured to select image processing for the captured image P1 executed by the generation unit 22c and to select a display pattern in the selected image processing. The setting unit 22a stores in advance an option including at least one of the suspension body 30, the landing target surface 31, and the obstacle 33 as a candidate for a subject whose visibility is relatively high. The setting unit 22a of this embodiment further stores options for the type of image processing included in the options that relatively increase the visibility of the subject, and options for the display pattern in each image processing. ..

Specifically, the setting unit 22a is provided on, for example, the outer edge portion 30a of the suspension body 30, the outer surface portion 30b of the suspension body 30, and the suspension body 30 as candidates for a subject whose visibility is relatively high. The engaging metal fitting 30c, the outer edge portion 31a of the landing target surface 31, the outer surface portion 31b of the landing target surface 31, the engaged metal fitting 31c provided on the landing target surface 31, the outer edge portion 33a of the obstacle 33, Options including the outer surface portion 33b of the obstacle 33 and the like are stored. As a candidate for a subject whose visibility is relatively high, the setting unit 22a may include a subject other than the above-mentioned candidate.

The outer edge portion 30a of the suspension body 30 described above is a portion (edge or the like) showing the outline of the suspension tool 7 or the suspension load (container 32). The outer surface portion 30b of the suspension body 30 is an upper surface portion or a side surface portion exposed to the outside of the suspension tool 7 or the suspended load. The engaging metal fittings 30c provided on the hanging body 30 engage with the engaged metal fittings 31c provided at the upper four corners of the container 32 when the container 32 is gripped by the hanging tool 7. It is a metal fitting (so-called twist lock pin) protruding from the bottom. In this embodiment, the engaging metal fittings 30c are provided at four places at the four corners of the hanger 7 and at four places at the center of the hanger 7, respectively, but they are engaged only at the four places at the four corners of the hanger 7. An alloy metal fitting 30c may be provided.

The outer edge portion 31a of the landing target surface 31 is the upper surface of the load (container 32) to be loaded and unloaded arranged on the land 36 and the ship 35, the floor surface of the land 36 and the ship 35, and the loading platform of the carrier 34. It is a portion showing the contour of a target surface on which the suspension body 30 is to be landed, such as the upper surface. The outer surface portion 31b of the landing target surface 31 is a surface portion of the target surface on which the above-mentioned suspension body 30 is landed. The engaged metal fitting 31c of the landing target surface 31 is a metal fitting (also referred to as a corner metal fitting) for engaging the engaging metal fittings 30c provided at the upper four corners of the container 32 to be handled.

The outer edge portion 33a of the obstacle 33 is a portion (edge or the like) showing the contour of the obstacle 33 with which the suspension body 30 may come into contact with or collide. The outer surface portion 33b of the obstacle 33 is a surface portion where the suspension body 30 of the obstacle 33 may come into contact with or collide with the hanger.

As a candidate for image processing that relatively enhances the visibility of the selected subject, the setting unit 22a is subjected to, for example, a process that further emphasizes the outer edge of the selected subject with respect to the captured image P1. Image processing, image processing that superimposes an additional marking 50 that emphasizes the position or shape of each selected subject on the captured image P1, and visibility of other subjects that are not selected compared to the captured image P1. The options of the type of image processing including the image processing that reduces the number of images are stored.

As an image process for superimposing the above-mentioned additional marking 50 on the captured image P1, for example, an image process for displaying an emphasis line 51 (image line) superimposed on the outer edge portions 30a, 31a, 33a of the selected subject. Image processing for displaying the colored portion 52 on the outer surface portions 30b, 31b, 33b, the engaging metal fitting 30c, the engaged metal fitting 31c, etc. of the selected subject, and a mark indicating the position of the selected subject. Options such as image processing for displaying 53 on or near the selected subject are stored.

Image processing that reduces the visibility of other subjects includes, for example, image processing that blurs the outer edge of the other subject, image processing that blurs the area in which the other subject appears, and image processing in which the other subject appears. Options such as image processing to increase transparency and image processing (masking) to cover other areas in which the subject is captured are stored.

Further, in the setting unit 22a, as items for setting a display pattern in image processing, for example, the degree of emphasizing the outer edge portion (edge) of the subject, the line type, color scheme, shape, and other visual recognition of the subject of the additional marking 50. Options for setting the degree of deterioration (resolution and transmittance) are stored.

According to the options stored in the setting unit 22a, the operator inputs data to the setting unit 22a using the input means 24 to select a subject whose visibility is relatively high, and an image to be executed by the generation unit 22c. The configuration is such that processing can be selected and display patterns for image processing can be selected.

In this embodiment, in the setting unit 22a, the outer edge portion 30a and the engaging metal fitting 30c of the hanging body 30, the outer edge portion 31a and the engaged metal fitting 31c of the landing target surface 31, are used as subjects for relatively increasing visibility. The case where the outer edge portion 33a and the outer surface portion 33b of the obstacle 33 are selected will be illustrated. Further, as the image processing to be executed by the generation unit 22c, an image process for superimposing the additional marking 50 on the captured image P1 was selected, and as the additional marking 50, the highlighted line 51, the coloring portion 52, and the mark 53 were selected. Illustrate the case. In the following, the data of the subject with relatively high visibility, the data of the image processing executed by the generation unit 22c, and the data of the display pattern of the image processing selected by the setting unit 22a are selected data relating to the subject, respectively. Selection data related to image processing and selection data related to display patterns.

The selection data related to the subject selected by the setting unit 22a is input to the position information acquisition unit 22b and the generation unit 22c, respectively, and the selection data related to image processing and the selection data related to the display pattern are input to the generation unit 22c. There is. The subject options, image processing options, and display pattern options stored in the setting unit 22a are not limited to the illustrated options, and other options may be stored.

The position information acquisition unit 22b is a photographed image P1 of a subject selected in advance by the setting unit 22a based on the photographed image P1 input from the photographing device 21 and the selection data regarding the subject input from the setting unit 22a. The two-dimensional coordinates in are sequentially acquired. The two-dimensional coordinates of the subject acquired by the position information acquisition unit 22b are sequentially input to the generation unit 22c.

The position information acquisition unit 22b of this embodiment acquires the two-dimensional coordinates of the captured image P1 of each subject selected by the setting unit 22a by performing image analysis of the captured image P1 input from the photographing device 21. It is configured. The position information acquisition unit 22b of this embodiment includes an outer edge portion 30a and an engaging metal fitting 30c of the hanging body 30 selected in the setting unit 22a, an outer edge portion 31a of the landing target surface 31, and an engaged metal fitting 31c, and an obstacle. The two-dimensional coordinates of the outer edge portion 33a and the outer surface portion 33b of the object 33 (horizontal beam 2c) in the captured image P1 are acquired.

The image analysis programs and software constituting the position information acquisition unit 22b are not particularly limited, and various programs and software can be applied. Specifically, the position information acquisition unit 22b can be configured by, for example, a program or software that uses pattern matching, feature-based matching, or the like, which are known image analysis techniques.

The position information acquisition unit 22b is not limited to the above configuration as long as it can acquire the two-dimensional coordinates of the subject in the captured image P1, and may have other configurations. The position information acquisition unit 22b acquires, for example, the three-dimensional position coordinates of the object to be the subject in the photographed image P1 by a sensor or a measuring device, and the acquired three-dimensional position coordinates and the photographing conditions (installation position and photographing) of the photographing device 21. It is also possible to calculate the two-dimensional coordinates of the subject in the captured image P1 based on the direction and the like).

The means for acquiring the three-dimensional position coordinates of the object to be the subject is, for example, a three-dimensional laser sensor installed in the trolley 5 that recognizes the shape of the subject such as the suspension body 30, the landing target surface 31, and the obstacle 33. It can be configured by a runout angle sensor or the like that detects the runout angle of the suspension body 30 with respect to the trolley 5. The above-mentioned means are also installed in, for example, a GPS receiver installed in the hanging tool 7, a sensor for measuring the feeding amount and the winding amount of the wire 6 that suspends the hanging tool 7, and the hanging tool 7. It can also be configured with a three-dimensional laser sensor or the like. Further, the above-mentioned means uses, for example, a stereo camera that captures a plurality of images including the suspension body 30 and three-dimensional position information of an object based on the plurality of images captured by the stereo camera by using a matching technique. It can also be configured with a position detection device to detect.

The generation unit 22c is based on the selection data regarding the subject input from the setting unit 22a, the selection data regarding image processing, the selection data regarding the display pattern, and the two-dimensional coordinates of the subject input from the position information acquisition unit 22b. , Image processing is performed on the captured image P1. Then, the setting unit 22a generates the processed image P3 in which the visibility of the subject selected in advance is relatively higher than the visibility of other subjects that have not been selected. The generation unit 22c may be configured to process the captured image P1 to generate the processed image P3, or generate the processed image P3 composed of a plurality of layers in which the upper layer image P2 is superimposed on the captured image P1. It can also be configured.

As illustrated in FIG. 4, in this embodiment, the generation unit 22c generates an upper layer image P2 including an additional marking 50 that emphasizes the position or shape of each of the predetermined subjects. Then, as illustrated in FIG. 5, a processed image P3 is generated by superimposing the generated upper layer image P2 on the captured image P1.

As illustrated in FIGS. 4 and 5, in the upper layer image P2 generated in this embodiment, the outer edge portion 30a of the suspension body 30 and the outer edge portion 31a of the landing target surface 31 are shown as additional markings 50. An emphasis line 51 that emphasizes the outer edge portion 33a of the obstacle 33 is included. Further, in the upper layer image P2, as an additional marking 50, the engaging metal fitting 30c, the engaged metal fitting 31c, and the outer surface portion 33b of the obstacle 33 are colored and emphasized with a color different from the color in the captured image P1. A colored portion 52 to be formed and a mark 53 indicating the center position of the landing target surface 31 are included.

In FIGS. 4 and 5, the emphasized line 51 is indicated by a thick line, the colored portion 52 is indicated by a shaded portion, and the mark 53 indicating the center position of the landing target surface 31 is indicated by a combination of a white circle and a crosshair. ing. The additional marking 50 may be a marking that can be easily distinguished from each other by making the line thickness, line type, color scheme, etc. different for each subject. The shape, color scheme, display size, and the like of the mark 53 indicating the center position of the landing target surface 31 are not limited to this embodiment, and may be, for example, a polygon, a double circle, an arrow, or the like. The processed image P3 generated by the generation unit 22c is configured to be sequentially input to the display control unit 22d.

The display control unit 22d sequentially displays the processed image P3 input from the generation unit 22c on the display device 23. The display device 23 is a monitor on which the processed image P3 output from the processing device 22 is displayed. The display device 23 of this embodiment is composed of a touch panel type monitor having a built-in input means 24.

The input means 24 is a means for inputting data to the setting unit 22a, and is composed of, for example, a touch panel, a touch pad, a button, a joystick, a mouse, a keyboard, and the like. The input means 24 of this embodiment is composed of a touch panel built in the display device 23. The input means 24 can also be attached to the operating device 13.

Next, an operation support method for the crane 1 using the operation support system 20 will be described.

Before starting the cargo handling work by the crane 1, the operator operates the input means 24 as a preset of the operation support system 20 to select a subject whose visibility is relatively high, and an image to be executed by the generation unit 22c. The processing is selected and the display pattern in the image processing is selected. In this embodiment, when the operator taps the setting icon displayed on the screen of the display device 23, the subject stored in the setting unit 22a on the screen of the display device 23 and has a relatively high visibility. The options of the above, the options of the image processing, and the options of the display pattern in the image processing are displayed respectively.

When the operator touches the screen of the display device 23 to complete the selection of the subject, the image processing, and the display pattern for which the visibility is relatively high, the selection data regarding the selected subject is the position information acquisition unit 22b and the generation unit. It is input to 22c, and the selection data related to image processing and the selection data related to the display pattern are input to the generation unit 22c. This completes the pre-setting of the operation support system 20.

In this embodiment, the case where the operator performs the presetting of the operation support system 20 is illustrated, but the subject, the image processing, and the display pattern for which the visibility is relatively high are preset in advance. The data may be stored in the setting unit 22a in advance. In this case, the preset work described above can be omitted.

As illustrated in FIG. 6, when the pre-setting of the operation support system 20 is completed and the operator starts the operation of the crane 1 by the operation device 13, the photographing device 21 photographs the photographed image P1 including the suspension body 30. The captured image P1 data is input to the generation unit 22c (S10). Next, the position information acquisition unit 22b acquires the two-dimensional coordinates of the captured image P1 of the subject selected in advance by the setting unit 22a, and the acquired two-dimensional coordinate data is input to the generation unit 22c (S20). ..

Next, the generation unit 22c selects in advance in the setting unit 22a based on the data of the captured image P1 input from the photographing device 21 and the data of the two-dimensional coordinates of the subject input from the position information acquisition unit 22b. A processed image P3 in which the visibility of the subject is relatively higher than the visibility of other unselected subjects is generated, and the generated processed image P3 is input to the display control unit 22d (S30). ).

Next, the display control unit 22d displays the processed image P3 input from the generation unit 22c on the display device 23 (S40). When step S40 is completed, the process returns to the start. By repeatedly and continuously executing the steps S10 to S40 described above in a predetermined cycle, the image on which the processed image P3 is continuously displayed is displayed on the display device 23 in a state close to real time. The above-mentioned steps S10 to S40 are repeatedly executed until the operator finishes the operation of the crane 1 by the operating device 13.

In this operation support system 20, even while steps S10 to S40 are repeatedly executed, the subject and image processing that relatively increase the visibility by performing the same operations as when the presets are performed, The system is such that the selection data related to the display pattern can be changed and updated. When the selection data is changed or updated, steps S10 to S40 are executed from the next cycle based on the updated selection data.

In this embodiment, further, the subject, image processing, and display pattern that relatively increase the visibility by tapping the display position of the subject displayed on the processed image P3 displayed on the screen of the display device 23 are related. The configuration is such that the selection data can be easily changed and updated.

Specifically, when the display position of a subject included in the selection data as a subject having relatively high visibility on the screen of the display device 23 is long-tapped, the subject is excluded from the selection data and is excluded from the selection data. If you long-tap the excluded subject, that subject will be added to the selection data as a subject with relatively high visibility. For example, when the operator wants to erase the additional marking 50 attached to the obstacle 33, the additional marking 50 attached to the obstacle 33 can be easily removed by long-tapping the display position of the obstacle 33. It can be erased.

Further, in this embodiment, when the display position of the subject included in the selection data is double-tapped on the screen of the display device 23, the type of image processing applied to the subject is changed. For example, from the image processing in which the operator superimposes the emphasis line 51 on the outer edge portion 30a of the suspension body 30 reflected in the captured image P1 on the suspension body 30, the photographed image P1 is processed and the suspension body 30 is displayed. If you want to change to image processing (sharpness) that emphasizes the outer edge 30a (edge), you can easily change the type of image processing applied to the hanging body 30 by double-tapping the display position of the hanging body 30. It has become.

Further, when the display position of the subject to which the image processing is applied is single-tapped on the screen of the display device 23, the display pattern of the image processing applied to the subject is changed. For example, when the operator wants to change the color of the line of the emphasis line 51 attached to the landing target surface 31, the line of the emphasis line 51 can be changed by single-tapping the display position of the landing target surface 31. You can easily change the color.

The operation method of the touch panel when changing or updating the selection data is not limited to the above-mentioned operation method, and can be set to another operation method. That is, for example, the operation of changing the subject having a relatively high visibility can be performed by a double tap or a single tap instead of a long tap. Further, in this embodiment, the case where the input means 24 is configured by the touch panel is illustrated. For example, the input means 24 is configured by a mouse or a touch pad, and the cursor displayed on the processed image P3 is a subject. By moving it up and clicking it, you can change or update the selected data.

As described above, according to the operation support system 20, among the subjects captured in the captured image P1 captured by the photographing device 21, at least one of the suspension body 30, the landing target surface 31, or the obstacle 33 is included in advance. The display device 23 displays the processed image P3 in which the visibility of the selected subject is higher than the visibility of the other non-selected subjects. As a result, the suspension body 30, the landing target surface 31, and obstacles that the operator operating the crane 1 needs to recognize when performing cargo handling work, as compared with the case where the captured image P1 is displayed as it is on the display device 23. It becomes easier to grasp the position and shape of a specific subject such as the object 33. Therefore, the workability of the operation of the crane 1 by the operator can be improved. Further, the operator can narrow down the subject to be watched during the cargo handling work, and it is not necessary to use more nerves than necessary for the subject that needs to be recognized, so that the operator can be made lighter.

As in this embodiment, the processed image P3 is an image in which the visibility of at least one of the outer edge portion 30a, the outer surface portion 30b, or the engaging metal fitting 30c of the suspension body 30 is enhanced as compared with the captured image P1. , It becomes very easy to grasp the position and shape of the suspension body 30, which is highly necessary to be recognized in cargo handling work. Therefore, it is advantageous to improve the workability of the operation of the crane 1. The portion of the suspension body 30 for enhancing visibility and the type of image processing are not limited to this embodiment and can be appropriately set. For example, the portion of the hanging body 30 that enhances visibility can be set only at the four corners of the hanging body 30, or the colored portion 52 may be superimposed on the outer surface portion 30b of the hanging body 30 to be displayed. can.

When the processed image P3 is an image in which the visibility of at least one of the outer edge portion 31a, the outer surface portion 31b, or the engaged metal fitting 31c of the landing target surface 31 is improved as compared with the captured image P1, in the landing operation. It becomes very easy to grasp the position and shape of the landing target surface 31, which is highly necessary to be recognized. Therefore, it is advantageous to improve the workability of the landing operation. The portion of the landing target surface 31 that enhances visibility and the type of image processing are not limited to this embodiment and can be appropriately set. For example, the portion of the landing target surface 31 that enhances visibility can be set only at the four corners of the landing target surface 31, or the colored portion 52 is superimposed on the outer surface portion 31b of the landing target surface 31 and displayed. It can also be configured to.

As in this embodiment, if the processed image P3 is an image in which the visibility of the engaging metal fitting 30c of the suspension body 30 and the engaged metal fitting 31c of the landing target surface 31 is improved as compared with the captured image P1, the operator. Since it becomes easier to grasp the relative positional relationship between the engaging metal fitting 30c and the engaged metal fitting 31c, it becomes easier to align the engaging metal fitting 30c and the engaged metal fitting 31c in the landing operation. Therefore, it becomes more and more advantageous to improve the workability of the landing operation.

In the captured image P1, it is difficult to distinguish between the obstacle 33 that the suspension body 30 may contact and collide with and another subject that the suspension body 30 does not contact and collide with. As described above, when the processed image P3 is an image in which the visibility of the outer edge portion 33a or the outer surface portion 33b of the obstacle 33 is enhanced, it becomes very easy to grasp the position and shape of the obstacle 33 to be noted. Therefore, it is advantageous to avoid contacting and colliding the suspension body 30 with the obstacle 33 during the cargo handling work.

When the processed image P3 is an image in which an additional marking 50 that emphasizes the position or shape of a subject selected in advance is superimposed on the captured image P1, the shape or position of the outer edge portion, outer surface portion, metal fitting, etc. of the subject is displayed. Since it can be clearly indicated by the additional marking 50, it becomes very easy to grasp the shape and position of the subject selected in advance. Further, if the display patterns such as the line type and the color scheme of the additional marking 50 attached to each subject are different, it becomes easier to distinguish the subjects whose visibility is enhanced.

FIG. 7 illustrates another embodiment of the operation support system 20 of the present invention.

As illustrated in FIG. 7, the generation unit 22c of the processing device 22 of this embodiment is selected in advance by performing image processing that reduces the visibility of a subject that has not been selected as compared with the captured image P1. The processed image P3 is generated in which the visibility of the subject is relatively higher than the visibility of other unselected subjects. More specifically, in this embodiment, the captured image P1 is subjected to image processing (image processing for lowering the resolution, etc.) that blurs a range in which a preselected subject is not captured, and other subjects that are not selected. Is made inconspicuous to generate a processed image P3 that makes it easier to visually identify a relatively preselected subject. In FIG. 7, a predetermined subject is shown by a solid line, and other subjects with blurring are shown by a broken line.

As image processing that reduces the visibility of other unselected subjects, in addition to image processing that blurs, image processing that enhances transparency, image processing that masks (masking), image processing that inserts mosaics, etc. It can be illustrated. The image processing for reducing the visibility of the subject may be performed by a method of processing the captured image P1 or by superimposing the upper layer image P2 on the captured image P1.

As described above, if the processed image P3 is an image in which the visibility of other subjects that are not selected is reduced as compared with the captured image P1, the information that the operator can see during the cargo handling work is being obtained during the cargo handling work. It is possible to narrow down to subjects that are highly necessary to be visually recognized. Therefore, it is advantageous to reduce the labor of the operator.

In this embodiment, as additional markings 50 that emphasize the position or shape of the subject selected in advance, the positions of the engaging metal fittings 30c of the hanging body 30 and the engaged metal fittings 31c of the landing target surface 31 are shown. The mark 53 is displayed on the processed image P3. In FIG. 7, the mark 53 is illustrated by a black arrow, but the shape and color of the mark 53 are not limited to this embodiment, and various figures and color schemes can be applied. In this way, if the processed image P3 is configured to display the mark 53 indicating the position of the subject, it becomes very easy for the operator to grasp the position of the subject.

In the above-described embodiment, various image processes are exemplified as image processes for relatively increasing the visibility of a preselected subject, but the combination of image processes executed by the generation unit 22c is the above-described embodiment. It is not limited to the form and can be combined as appropriate. For example, the processed image P3 is generated by applying both image processing that emphasizes the outer edge of a preselected subject and image processing that blurs other unselected subjects to the captured image P1. You can also. Further, if the image processing is such that the visibility of the preselected subject is relatively higher than the visibility of other unselected subjects, the image processing is not limited to the image processing exemplified above, and other images. Processing can also be applied to generate the processed image P3.

In the above-described embodiment, the operation support system 20 applied to the quay crane has been illustrated, but the operation support system 20 is not limited to the quay crane, and handles the cargo of the container 32 with respect to the transporter and the storage lane at the container terminal. It can also be applied to gated cranes and overhead cranes. Further, the operation support system 20 is not limited to the container crane that handles the cargo of the container 32, and can be applied to various cranes that handle the cargo of other cargo handling objects.

1 Crane 2 Leg structure 2c Horizontal beam 5 Trolley 7 Suspension tool 20 Operation support system 21 Imaging device 22 Processing device 22a Setting unit 22b Position information acquisition unit 22c Generation unit 22d Display control unit 23 Display device 24 Input means 30 Suspension body 31 Arrival Floor Target surface 32 Container 33 Obstacle 50 Additional marking P1 Captured image P2 Upper layer image P3 Processed image

Claims (7)

In a crane operation support system equipped with a photographing device that captures a captured image including a suspended body suspended from a trolley and a display device that displays an image based on the captured image.
The imaging device and the processing device connected to the display device are provided.
The processing device includes at least one of the suspended body, the landing target surface on which the suspended body is landed, or an obstacle existing on the cargo handling path of the suspended body among the subjects reflected in the captured image. A processed image in which the visibility of the subject selected in advance is relatively higher than the visibility of the other unselected subjects is generated, and the generated processed image is displayed on the display device. A crane operation support system characterized by its configuration. The processed image is an image in which the visibility of at least one of the outer edge portion of the hanging body, the outer surface portion of the hanging body, and the engaging metal fitting provided on the hanging body is enhanced as compared with the captured image. The crane operation support system according to claim 1. The processed image is visually recognized as at least one of the outer edge portion of the landing target surface, the outer surface portion of the landing target surface, or the engaged metal fitting provided on the landing target surface as compared with the captured image. The crane operation support system according to claim 1 or 2, which is an image with enhanced properties. The crane operation support system according to any one of claims 1 to 3, wherein the processed image is an image in which the visibility of the outer edge portion or the outer surface portion of the obstacle is enhanced as compared with the captured image. The crane operation support according to any one of claims 1 to 4, wherein the processed image is an image in which an additional marking emphasizing the position or shape of the preselected subject is superimposed on the captured image. system. The crane operation support system according to any one of claims 1 to 5, wherein the processed image is an image in which the visibility of the other subject is reduced as compared with the captured image. In a crane operation support method in which a photographed image including a suspended body suspended from a trolley is photographed by a photographing device and an image based on the photographed image captured by the photographing device is displayed on a display device.
Based on the step of photographing the photographed image including the suspended body by the photographing apparatus, the step of acquiring the two-dimensional coordinates of the subject reflected in the photographed image in the photographed image, and the acquired two-dimensional coordinates. Among the subjects, the visibility of the predetermined subject including at least one of the suspension body, the landing target surface on which the suspension body is landed, or an obstacle existing on the cargo handling path of the suspension body. It is characterized by including a step of generating a processed image that is relatively higher than the visibility of the other unselected subject, and a step of displaying the generated processed image on the display device. How to support the operation of the crane.

Images