EP4729459A1

EP4729459A1 - Crane device

Info

Publication number: EP4729459A1
Application number: EP24825763.6A
Authority: EP
Inventors: Keita MASHIMA; Hiroyuki Hayashi
Original assignee: Tadano Ltd
Current assignee: Tadano Ltd
Priority date: 2023-06-19
Filing date: 2024-06-10
Publication date: 2026-04-22
Also published as: WO2024262357A1; JP2025000438A

Abstract

[Problem] To provide a crane apparatus that can eliminate a blind spot irrespective of a type or specifications of the apparatus or can eliminate the blind spot in accordance with a request of an operator.[Solutions] A crane apparatus 12 includes a safety device 17. The safety device 17 includes a wide-angle camera 18, a display 84 installed in a cabin, and a controller 70. The wide-angle camera 18 is installed on an upper surface of a boom, and takes an area around a tip end portion of the boom, an area facing the upper surface of the boom, side areas of the boom, and a rear area of the cabin, as a capture range. The controller 70 determines a kind of a surveillance image displayed on the display 84, based on first designation information stored in a memory 72 in accordance with specifications and second designation information indicating a screen designation by an operator, and generates surveillance image data indicating the surveillance image of the determined kind, based on the captured image data output by the wide-angle camera 18.

Description

TECHNICAL FIELD

The present invention relates to a crane apparatus including a boom that can slew, stand, lie, extend, and retract.

BACKGROUND ART

Patent Document 1 discloses a crane apparatus including a slewing base and a boom mounted on the slewing base. The boom can stand, lie, extend, and retract. A sensor is provided at a tip end portion of the boom as a safety device. The sensor detects an obstacle approaching the tip end portion of the boom.

PRIOR ART DOCUMENT

PATENT DOCUMENT

[Patent Document 1] International Publication WO2022/092093

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

By the way, types of the crane apparatuses include a fixed type such as a tower crane as well as a mobile type such as a rough terrain crane and an all-terrain crane. Even among the crane apparatuses of the same type, a size, a working radius, or the like are different, and an area that becomes a blind spot for an operator at work varies, depending on specifications (model). For example, an area on an upper surface side of the boom becomes the blind spot in a model, an area in a periphery of a tip end of the boom becomes the blind spot in another model, and furthermore both of these areas become the blind spots in some cases. In this manner, the area that becomes the blind spot to be surveilled in the boom varies depending on the model of the crane apparatus.
To ensure safety of a work, measures by which the operator can surveil the above-described area and that eliminate the blind spot are needed. However, it is difficult to surveil the blind spot that varies depending on the specifications of the crane apparatus, using the above-described sensor. Moreover, there is a reason that the above-described area (blind spot to be eliminated) differs also depending on a request of the operator who actually operates the boom.
The present invention has been made in view of such a background, and an object thereof is to provide a crane apparatus that can eliminate a blind spot irrespective of a type or specifications of the apparatus, or can eliminate the blind spot in accordance with a request of an operator.

MEANS FOR SOLVING THE PROBLEMS

(1) A crane apparatus according to the present invention includes a slewing base, a cabin mounted on the slewing base and in which an operating device is installed, a boom provided to the slewing base so as to be able to stand and lie, the boom being able to extend and retract, and a safety device having a wide-angle camera attached to an upper surface of the boom in a lie-down posture, a controller, and a display. The controller is configured to execute generation processing of generating surveillance image data based on captured image data generated by the wide-angle camera, and display processing of causing the display to display a surveillance image indicated by the surveillance image data.

Since the wide-angle camera is attached to the upper surface of the boom, the camera can capture an area around a tip end portion of the boom, side areas of the boom, an area facing the upper surface of the boom, and a rear area being surroundings of a base end of the boom together. The controller generates the surveillance image data from the captured image data with respect to an area indicated by a setting prestored in a memory or an area designated by an operator, for example (generation processing), and inputs the surveillance image data to a display to cause the display to display the surveillance image (display processing). Therefore, even if an area that becomes a blind spot, the area being an area determined by a type or specifications or an area requested by the operator, is the area around the tip end of the boom, the side areas of the boom, the area facing the upper surface of the boom, or the rear area around the base end of the boom, the safety device can cause the display to display the surveillance image that captures the area. Therefore, the crane apparatus according to the present invention can eliminate the blind spot irrespective of the type or the specifications, or can eliminate the blind spot in accordance with a request of the operator.
(2) The controller may be configured to further execute designation information obtaining processing of obtaining designation information. The generation processing is one or more processes indicated by the designation information and selected from among first processing of generating boom tip-end surveillance image data by correcting the captured image data based on a first correction reference in accordance with a tip end of the boom captured in a captured image indicated by the captured image data, second processing of generating boom side surveillance image data by correcting the captured image data based on a predetermined second correction reference in an area captured in the captured image indicated by the captured image data, the area being on a side of the boom, third processing of generating boom upper-surface side surveillance image data by correcting the captured image data based on a predetermined third correction reference in an area captured in the captured image indicated by the captured image data, the area facing the upper surface of the boom, and fourth processing of generating rear surveillance image data by correcting the captured image data based on a fourth correction reference in accordance with a base end of the boom captured in the captured image indicated by the captured image data.
The designation information is prestored in the memory of the controller, or the operator inputs the information to the controller, for example. The controller obtains the designation information by reading the designation information from the memory or accepting an input of the designation information by the operator. The controller executes processing designated by the designation information among the first processing to the fourth processing as the generation processing to generate the surveillance image data. Therefore, an appropriate surveillance image in accordance with the type or the specifications of the crane apparatus is displayed on the display, or an appropriate surveillance image designated by the operator is displayed on the display.
(3) The crane apparatus according to the present invention may further include a derricking angle sensor configured to output a first detection value indicating a derricking angle of the boom. The fourth processing includes processing of correcting the captured image data based on the fourth correction reference, and transformation processing of transforming a viewpoint based on the derricking angle indicated by the first detection value. A rear surveillance image indicated by the rear surveillance image data is an overhead view image indicating a rear of the cabin.
A position and a height of the wide-angle camera taking the slewing base as a reference are changed by derricking of the boom. The transformation processing for transforming into the overhead view image is performed based on the derricking angle of the boom, the angle being a parameter that determines the position and the height of the wide-angle camera. Therefore, the rear surveillance image always overlooking the rear of the cabin from a fixed position is displayed on the display, not depending on the derricking angle of the boom.
(4) The crane apparatus according to the present invention may further include a derricking angle sensor configured to output a first detection value indicating a derricking angle of the boom. The second processing further includes processing of correcting the captured image data based on the second correction reference, and rotation processing of rotating a boom side surveillance image that is a surveillance image indicated by the boom side surveillance image data, by a rotation angle in accordance with the derricking angle indicated by the first detection value.
The boom side surveillance image is tilted in accordance with the derricking of the boom. The controller causes the boom side surveillance image to rotate in accordance with the derricking angle of the boom to rectify a tilt of the boom side surveillance image. Therefore, the crane apparatus according to the present invention allows the operator to easily recognize obstacles compared to a case where the tilt of the boom side surveillance image is not rectified.
(5) The crane apparatus according to the present invention may further include a derricking angle sensor configured to output a first detection value indicating a derricking angle of the boom. The controller is configured to execute the third processing based on a fact that the designation information indicates an execution of the third processing, and execute first enlargement processing of enlarging a boom upper-surface side surveillance image indicated by the boom upper-surface side surveillance image data generated by the third processing, based on a fact that the boom is determined to be in a process of standing up by a change over time of the derricking angle indicated by the first detection value, before executing the display processing.
When the boom is in the process of standing up, there occurs a risk of a contact between the upper surface of the boom and an obstacle. When the boom is the process of standing up, the controller enlarges and displays the boom upper-surface side surveillance image. Therefore, the crane apparatus according to the present invention allows the operator to recognize the obstacle more easily compared to a case where the display is caused to always display the boom upper-surface side surveillance image at a fixed size.
(6) The crane apparatus according to the present invention may further include a slewing angle sensor configured to output a second detection value indicating a slewing angle of the slewing base. The boom side surveillance image data includes boom left-side surveillance image data indicating a left side of the boom and boom right-side surveillance image data indicating a right side of the boom. The controller is configured to execute the second processing based on a fact that the designation information indicates an execution of the second processing, execute second enlargement processing of enlarging a boom left-side surveillance image indicated by the boom left-side surveillance image data generated by the second processing, based on a fact that a left slew is determined by the second detection value, before executing the display processing, and execute third enlargement processing of enlarging a boom right-side surveillance image indicated by the boom right-side surveillance image data generated by the second processing, based on a fact that a right slew is determined by the second detection value, before executing the display processing.
When the boom is slewed, there occurs a risk of a contact between a side surface of the boom and an obstacle. The controller enlarges and displays the boom left-side surveillance image when the boom is slewed left, and enlarges and displays the boom right-side surveillance image when the boom is slewed right. Therefore, the crane apparatus according to the present invention allows the operator to recognize the obstacle more easily compared to a case where the display is caused to always display the boom side surveillance image at a fixed size.
(7) The boom may have a plurality of tube bodies including a base boom. The wide-angle camera is provided to the base boom.
Since the wide-angle camera is provided to the base boom whose position is not changed by an extension or a retraction of the boom, a wired connection between the wide-angle camera and the controller becomes easier compared to a case where the wide-angle camera is provided to another tube body whose position is changed by the extension and the retraction of the boom.
(8) The crane apparatus according to the present invention may further include a boom length sensor configured to output a third detection value indicating an extension length of the boom. The boom has a plurality of tube bodies including a base boom. The wide-angle camera is provided to the base boom. The first processing includes processing of correcting the captured image data based on the first correction reference, and fourth enlargement processing of enlarging by an enlargement ratio in accordance with the third detection value.
In a configuration where the wide-angle camera is provided to the base boom, the tip end of the boom leaves the wide-angle camera as the boom extends. A boom tip-end surveillance image that is a surveillance image indicated by the boom tip-end surveillance image data is enlarged by the enlargement ratio in accordance with the extension length of the boom, which allows the operator to easily recognize an obstacle approaching the tip end of the boom.
(9) An angle of view of the wide-angle camera may be equal to or larger than 180 degrees.
The wide-angle camera having the angle of view equal to or larger than 180 degrees is used.

EFFECTS OF THE INVENTION

The crane apparatus according to the present invention can eliminate a blind spot irrespective of a type or specifications of the apparatus, or can eliminate the blind spot in accordance with a request of an operator.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a functional block of a crane apparatus 12.
Fig. 2 is a schematic side view of a crane vehicle 10 in a state where a boom 22 is caused to stand up.
Fig. 3 is a schematic side view of the crane vehicle 10 in a lie-down posture in which the boom 22 lies down.
Fig. 4 is a diagram showing an operating device 29 in a cabin 13.
Fig. 5 is a flowchart of surveillance image displaying processing.
Fig. 6(A) is a schematic diagram of a captured image captured by a wide-angle camera 18, Fig. 6(B) is a diagram showing a rear surveillance image, Fig. 6(C) is a diagram showing a boom left-side surveillance image, Fig. 6(D) is a diagram showing a boom right-side surveillance image, Fig. 6(E) is a diagram showing a boom tip-end surveillance image, and Fig. 6(F) is a diagram showing a boom upper-surface side surveillance image.

MODES FOR CARRYING OUT THE INVENTION

Hereinafter, an embodiment of the present invention will be described. Note that it goes without saying that the embodiment described below is merely an example of the present invention, and the embodiment can be modified as appropriate without departing from the gist of the present invention. For example, an execution order of each processing described later can be changed as appropriate without departing from the gist of the present invention. Or, a part of processing described later may be omitted as appropriate without departing from the gist of the present invention.
Fig. 1 is a functional block diagram of a crane apparatus 12. Fig. 2 is a side view of the crane vehicle 10 in a state where the boom 22 stands up. Fig. 3 is a schematic side view of the crane vehicle 10 in a state where the boom 22 lies down.
As shown in Figs. 2 and 3, the crane vehicle 10 is a rough terrain crane. However, the crane vehicle 10 may be an all-terrain crane.
The crane vehicle 10 includes a traveling body 11, an outrigger device 14 attached to the traveling body 11, and the crane apparatus 12 and a cabin 13 that are mounted on the traveling body 11. Note that an illustration of the outrigger device 14 is omitted in Fig. 3.
The outrigger device 14 has a plurality of jacks 24 that contact with the ground to stabilize a posture of the crane vehicle 10. The crane apparatus 12 is used in a state where the jacks 24 are extended and the posture of the crane vehicle 10 is stabilized.
The crane apparatus 12 has a slewing base 21, the boom 22, a winch 23, a group of sensors 26 (see Fig. 1), a group of hydraulic actuators 27 (see Fig. 1), a hydraulic pressure supplying device 28 (see Fig. 1), an operating device 29 (see Fig. 1), a hook block 32, and a safety device 17 (see Fig. 1).
The slewing base 21 is held by the traveling body 11 so as to be able to slew. A rotation axis of the slewing base 21 extends along a vertical direction (up-down direction), and is located approximately at a center of the traveling body 11.
The boom 22 is supported by the slewing base 21 so as to be able to stand and lie. In other words, the boom 22 can stand, lie, and slew.
The boom 22 includes a base boom 42, an intermediate boom 43, and a top boom 44. Each of the base boom 42, the intermediate boom 43, and the top boom 44 has a rectangular tube shape. The intermediate boom 43 is located inside the base boom 42, and the top boom 44 is located inside the intermediate boom 43. In other words, the base boom 42, the intermediate boom 43, and the top boom 44 are arranged in a nested manner, and form a so-called telescopic structure. The boom 22 extends and retracts when the intermediate boom 43 slides relative to the base boom 42 and the top boom 44 slides relative to the intermediate boom 43. The base boom 42, the intermediate boom 43, and the top boom 44 correspond to "tube bodies" recited in the claims. Note that the boom 22 may be configured by two tube bodies, or may be configured by four or more tube bodies.
The base boom 42 is located at an outermost. Therefore, outer surfaces of the base boom 42 are always exposed to an outside. The base boom 42 having the rectangular tube shape has an upper surface 45 that serves as an upper surface in a lie-down posture (see Fig. 3) in which the boom 22 extends along a horizontal direction. A wide-angle camera 18 described later is provided to the upper surface 45.
The boom 22 is supported by the slewing base 21 at the base boom 42 so as to be able to stand and lie. In other words, the base boom 42 does not move (slide) by an extraction or a retraction of the boom 22. In other words, a position of the wide-angle camera 18 attached to the base boom 42 is not changed by the extraction or the retraction of the boom 22.
The winch 23 is attached to a base end of the boom 22 or the slewing base 21. The winch 23 has a drum 56 around which a wire rope 41 (hereinafter denoted as "a wire 41") is wound, and a sheave 57 around which the wire 41 is hung. The wire 41 is wound to the drum 56 or rolled out from the drum 56 by driving the winch 23. Note that illustrations of the winch 23, the wire 41, the sheave 57, the drum 56, and the like are omitted in Fig. 3.
The wire 41 is hung around the sheave 57 provided at a base end portion of the boom 22, a sheave 58 provided at a tip end portion of the boom 22, and a pulley device (not shown). The pulley device has a plurality of first sheaves (not shown) provided at the tip end portion of the boom 22 and a plurality of second sheaves (not shown) provided to the hook block 32. Note that an illustration of the hook block 32 is omitted in Fig. 3.
The hook block 32 includes a hook body 33, the plurality of second sheaves rotatably held in the hook body 33, and a hook 34 provided to a lower surface of the hook body 33.
Two winches 23 may be provided to the crane apparatus 12. The hook block 32 that is a so-called main hook is connected to the wire 41 of one winch 23, and a subhook (not shown) is connected to the wire 41 of the other winch 23.
As shown in Fig. 1, the group of hydraulic actuators 27 has a slewing motor 51, a derricking cylinder 52, an extending/retracting cylinder 53, a hydraulic motor 54, and the jacks 24.
The slewing motor 51 is a hydraulic motor that rotates via hydraulic oil supplied from the hydraulic pressure supplying device 28, and causes the slewing base 21 to slew. The derricking cylinder 52 is a hydraulic cylinder that extends and retracts via the hydraulic oil supplied from the hydraulic pressure supplying device 28, and causes the boom 22 to stand or lie. The extending/retracting cylinder 53 is a hydraulic cylinder that extends and retracts via the hydraulic oil supplied from the hydraulic pressure supplying device 28, and causes the boom 22 to extend or retract. The hydraulic motor 54 rotates via the hydraulic oil supplied from the hydraulic pressure supplying device 28, and causes the drum 56 of the winch 23 to rotate.
The hydraulic pressure supplying device 28 includes a hydraulic pump driven by an engine 15 mounted on the traveling body 11, pipes that connect the hydraulic pump with the slewing motor 51 or the like of the group of hydraulic actuators 27, and a hydraulic pressure switching valve and the like provided to the pipes and the like. The hydraulic pressure switching valve may be a so-called electromagnetic valve, and is driven by a drive signal input from a controller 70. By driving the electromagnetic valve, the slewing motor 51, the derricking cylinder 52, the extending/retracting cylinder 53, the hydraulic motor 54, and the jacks 24 are driven. Namely, the controller 70 can cause the boom 22 to slew, stand, lie, extend, and retract, wind up or pay out the wire 41, and cause the jacks 24 to extend or retract by outputting drive signals.
The group of sensors 26 has a slewing angle sensor 61, a boom length sensor 62, and a derricking angle sensor 63.
The slewing angle sensor 61 outputs a detection value in accordance with a slewing angle of the slewing base 21 from a slew reference position. The slew reference position is a position of a state shown in Fig. 3 in which the boom 22 protrudes forward. The slewing angle sensor 61 is a rotary encoder provided to a slewing axis of the slewing base 21, for example. The slewing angle sensor 61 outputs, as the detection value, pulse signals whose number is in accordance with the slewing angle of the slewing base 21. In the following, description will be made assuming that the slewing angle is a counter-clockwise angle from the slew reference position. In other words, the slewing angle is increased in a left slew of the boom 22, and is decreased in a right slew of the boom 22. The detection value output by the slewing angle sensor 61 corresponds to "a second detection value" recited in the claims.
The boom length sensor 62 outputs a detection value in accordance with a length of the boom 22. The detection value output by the boom length sensor 62 may indicate a length from the base end to the tip end of the boom 22, or may indicate a length by which the boom 22 extends. The boom length sensor 62 may be a sensor that outputs the detection value directly indicating the boom length, may be a sensor that outputs the detection value indicating an extension length of the extending/retracting cylinder 53, or may be a sensor that outputs the detection value indicating drive time of the extending/retracting cylinder 53. The detection value output by the boom length sensor 62 corresponds to "a third detection value" recited in the claims.
The derricking angle sensor 63 outputs a detection value in accordance with a derricking angle of the boom 22. The derricking angle sensor 63 is a tilt sensor that outputs an angle with respect to a horizontal surface or a horizontal sensor, for example. Or, the derricking angle sensor 63 may be a sensor that outputs the detection value indicating an extension length or drive time of the derricking cylinder 52. The detection value output by the derricking angle sensor 63 corresponds to "a first detection value" recited in the claims.
In the following, description will be made with also describing the detection value output by the slewing angle sensor 61, the boom length sensor 62, and the derricking angle sensor 63 as "a sensor detection value".
The slewing angle sensor 61, the boom length sensor 62, and the derricking angle sensor 63 are connected to the controller 70 by cables. In other words, the sensor detection values output by the slewing angle sensor 61 and the like are input to the controller 70. Note that the sensor detection value output by each sensor may be input to the controller 70 by wireless communication.
Each sensor such as the slewing angle sensor 61 performs detections continually at a predetermined time interval (so-called sampling period/sampling cycle) and outputs the sensor detection values at the predetermined time interval.
Fig. 4 is a diagram showing the operating device 29 in the cabin 13.
As shown in Fig. 4, the operating device 29 is arranged in the cabin 13. The operating device 29 includes operation levers, foot pedals, operation buttons, and the like that are operated by an operator.
The operating device 29 is connected to the controller 70 (see Fig. 1) by signal lines (not shown). The operator operates the operating device 29 to input instructions to the controller 70 and operates the crane apparatus 12.
As shown in Fig. 4, a control monitor device 80 is installed in the cabin 13. As shown in Fig. 1, the control monitor device 80 includes a display 81, a transparent sheet-like touch sensor 82 overlaid on the display 81, and a speaker 83. Namely, the control monitor device 80 is a so-called AML.
The operator inputs (selects) an image designation using the touch sensor 82. The image designation is information indicating one or more images among a rear surveillance image shown in Fig. 6(B), a boom left-side surveillance image shown in Fig. 6(C), a boom right-side surveillance image shown in Fig. 6(D), a boom tip-end surveillance image shown in Fig. 6(E), and a boom upper-surface side surveillance image shown in Fig. 6(F). The operator inputs designation of one or more images to be displayed on a display 84 among the five images through the touch sensor 82. The input image designation is stored in a memory 72 (see Fig. 1) as second designation information.
As shown in Fig. 1, the safety device 17 includes the wide-angle camera 18, the display 84, and the controller 70.
The wide-angle camera 18 includes a condenser lens, an optical system, a plurality of imaging elements, a control circuit, and a communication interface. The condenser lens is what is called a wide-angle lens or a fisheye lens, and the wide-angle camera 18 is what is called a wide-angle camera or a fisheye camera.
The optical system has a plurality of lenses and guides light collected by the condenser lens to the imaging elements, for example. The plurality of imaging elements includes image sensors such as CCDs and CMOS sensors. The plurality of imaging elements is arranged in a matrix manner (in a grid manner). The plurality of imaging elements outputs image data in accordance with incident light. The image data consists of a plurality of pieces of pixel data and pixel position data. If a captured image is a color image, one piece of the pixel data consists of three pieces of color pixel data, for example. The control circuit captures images by the wide-angle camera 18 and generates the image data. The communication interface converts the image data into transmission data and outputs the converted data. The control circuit is a driver IC, for example, and the communication interface is a communication IC, for example.
As shown in Figs. 2 and 3, the wide-angle camera 18 is attached to the upper surface 45 of the base boom 42. The wide-angle camera 18 is arranged apart from a base end of the base boom 42. In other words, the wide-angle camera 18 is located at a position higher than the slewing base 21 in a state where the boom 22 is caused to stand up (crane working state). Therefore, the rear surveillance image (see Fig. 6(B)) generated from the captured image by the wide-angle camera 18 captures a wide range in a rear of the cabin 13. In other words, by attaching the wide-angle camera 18 to the upper surface 45 of the boom 22, the rear of the cabin 13 can be displayed as an overhead view image in a range wider than that in a case where the wide-angle camera 18 is attached to the slewing base 21.
In the example shown in Figs. 2 and 3, the wide-angle camera 18 is arranged at a biased position on a tip end side of a center of the base boom 42 so as to be able to generate the rear surveillance image (see Fig. 6(B)) that captures a wider range. However, the wide-angle camera 18 may be arranged at the center of the base boom 42, or may be arranged at another biased position on a base end side of the center.
The wide-angle camera 18 may be a monochrome camera that outputs image data indicating monochrome images, or may be a color camera that outputs image data indicating color images. Furthermore, the image data output by the wide-angle camera 18 may be data indicating still images, or may be video image data consisting of a plurality of frames.
Dot-dash lines shown in Figs. 2 and 3 indicate a capture range (angle of view) of the wide-angle camera 18. As shown in Figs. 2 and 3, an attachment position of the wide-angle camera 18 and the angle of view of the wide-angle camera 18 are determined so that the tip end portion of the boom 22, left and right sides of the boom 22, an area facing the upper surface 45 of the boom 22, and the rear of the cabin 13 are within the capture range (angle of view) of the wide-angle camera 18. In other words, the wide-angle camera 18 regards all areas of an area around the tip end portion of the boom 22, an area on a left side of the boom 22, an area on a right side of the boom 22, the area facing the upper surface 45 of the boom 22, and a rear area of the cabin 13 as a capture area.
The angle of view of the wide-angle camera 18 is set to be equal to or larger than 180 degrees. In the example shown in Figs. 2 and 3, the angle of view of the wide-angle camera 18 is 220 degrees. Note that an angle of view of a wide-angle camera is generally 220 degrees at most. In other words, the angle of view of the wide-angle camera 18 is set in a range of 180 degrees or larger and smaller than 220 degrees.
As shown in Fig. 1, the wide-angle camera 18 and the controller 70 are connected by a cable 19. Captured image data output by the wide-angle camera 18 is input to the controller 70 through the cable 19. However, the captured image data may be input to the controller 70 by wireless communication. In this case, a battery and a sending antenna are attached to the boom 22, and a receiving antenna connected to the controller 70 by a communication line is installed in the cabin 13. The wide-angle camera 18 captures images and sends the captured image data from the sending antenna, by electric power supplied from the battery. The sent captured image data is received by the receiving antenna and is input to the controller 70.
The wide-angle camera 18 captures images continually at a predetermined time interval and outputs the captured image data at the predetermined time interval. Or, the wide-angle camera 18 continually outputs the video image data consisting of the plurality of frames.
As shown in Fig. 4, the display 84 is installed in the cabin 13. As shown in Fig. 1, the display 84 is connected to the controller 70 by a signal line. The display 84 displays an image indicated by image data input from the controller 70. Specifically, the display 84 displays the rear surveillance image shown in Fig. 6(B), the boom left-side surveillance image shown in Fig. 6(C), the boom right-side surveillance image shown in Fig. 6(D), the boom tip-end surveillance image shown in Fig. 6(E), and the boom upper-surface side surveillance image shown in Fig. 6(F). The display 84 corresponds to "a display" recited in the claims. Note that the five images may be displayed on the display 81 of the control monitor device 80. In this case, the display 81 corresponds to "a display" recited in the claims.
As shown in Fig. 1, the controller 70 includes a CPU 71 that is a central processing unit, the memory 72, a power supply circuit 74, and a communication bus (not shown). The controller 70 is realized by ICs, computers, resistors, diodes, coils, capacitors, and the like implemented on a control board, for example. The control board is arranged in a control box arranged in the cabin 13, for example.
The CPU 71, the memory 72, the hydraulic pressure supplying device 28, the slewing angle sensor 61 and the like of the group of sensors 26, the wide-angle camera 18, and the operating device 29 are connected to the communication bus (not shown). A later-described control program 75 executed by the CPU 71 reads data and information from the memory 72, causes the memory 72 to store data and information, and controls driving of the group of hydraulic actuators 27. Furthermore, the control program 75 obtains the sensor detection values output by the group of sensors 26, the captured image data output by the wide-angle camera 18, and inputs made by the operator to the control monitor device 80 and the operating device 29.
The memory 72 prestores an OS 77 that is an operating system, and the control program 75 to be executed by the CPU 71. Furthermore, the memory 72 prestores first designation information, and stores the second designation information indicating the image designation input by the operator through the control monitor device 80. The first designation information corresponds to "designation information" recited in the claims. The second designation information corresponds to "designation information" recited in the claims. The first designation information and the second designation information correspond to "designation information" recited in the claims.
As with the second designation information, the first designation information is information indicating one or more of the five images. A developer of the crane apparatus 12 causes the memory 72 to store the first designation information in accordance with specifications determined by a type, a model, a destination, or the like of the crane apparatus 12.
Furthermore, the memory 72 prestores a first correction function, a second correction function, a third correction function, a fourth correction function, a fifth correction function, a first transformation function, a second transformation function, and a third transformation function.
Fig. 6(A) is a schematic diagram of the captured image captured by the wide-angle camera 18. Fig. 6(B) is a schematic diagram showing the rear surveillance image that is an image that captures the rear of the cabin 13. Fig. 6(C) is a schematic diagram of the boom left-side surveillance image that captures the left side of the boom 22. Fig. 6(D) is a schematic diagram of the boom right-side surveillance image that captures the right side of the boom 22. Fig. 6(E) is a schematic diagram of the boom tip-end surveillance image that captures surroundings of the tip end portion of the boom 22. Fig. 6(F) is a schematic diagram of the boom upper-surface side surveillance image that captures the area facing the upper surface 45 of the boom 22.
The first correction function, the second correction function, the third correction function, the fourth correction function, the fifth correction function, the first transformation function, the second transformation function, and the third transformation function will be described with reference to Fig. 6.
The first correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate corrected image data showing the rear of the cabin 13. Specifically, the first correction function is a function that discards data of a portion of the captured image indicated by the captured image data, the portion being an area other than an area portion that becomes the rear of the cabin 13, and corrects a distortion. The first correction function discards data indicating an area other than surroundings of a fourth correction reference (see Fig. 6(A)), the area being an area other than an area surrounded by a two-dot-dash line in Fig. 6(A), and corrects the distortion.
The first correction function receives the captured image data and the derricking angle detected by the derricking angle sensor 63 as arguments and outputs the corrected image data as a return value. To describe in detail, a height of the wide-angle camera 18 is changed in accordance with the derricking angle of the boom 22. When the height of the wide-angle camera 18 is changed, a range captured in the captured image indicated by the captured image data is changed. The first correction function determines pixels in an area to be discarded, based on the received derricking angle. In other words, a range captured in an image indicated by the generated corrected image data becomes a constant range, not depending on the derricking angle of the boom 22.
The first transformation function is a function that changes a viewpoint in the image indicated by the corrected image data. The first transformation function is a function that performs a projective transformation, for example. The first transformation function receives the corrected image data and the derricking angle detected by the derricking angle sensor 63 as arguments and outputs rear surveillance image data. The rear surveillance image data is an image showing the rear of the cabin 13 and is an overhead view image. To describe in detail, the position (viewpoint) of the wide-angle camera 18 is changed in accordance with the derricking angle of the boom 22. The first transformation function changes the position of the viewpoint based on the received derricking angle. In other words, the rear surveillance image indicated by the generated rear surveillance image data becomes an overhead view image having a same viewpoint, not depending on the derricking angle of the boom 22.
The second correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom tip-end surveillance image data that captures a periphery of the tip end portion of the boom 22. Specifically, the second correction function is a function that discards data of a portion of the captured image indicated by the captured image data, the portion being an area other than an area portion of the periphery of the tip end portion of the boom 22 indicated by a first correction reference (see Fig. 6(A)), and corrects a distortion. The second correction function discards data indicating an area other than surroundings of the first correction reference (see Fig. 6(A)), the area being an area other than an area surrounded by dotted lines in Fig. 6(A), and corrects the distortion. The second correction function receives the captured image data as an argument and outputs the boom tip-end surveillance image data as a return value.
The second transformation function is a function that enlarges an image. The second transformation function receives image data and an enlargement ratio as arguments and outputs image data indicating an enlarged image. The second transformation function is an affine transformation (affine matrix) that performs an enlargement, for example.
The third correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom upper-surface side surveillance image data. The boom upper-surface side surveillance image data is data indicating the boom upper-surface side surveillance image that captures the area facing the upper surface 45 of the boom 22.
The third correction function is a function that discards data of a portion of the captured image indicated by the captured image data, the portion being an area other than an area portion facing the upper surface 45 of the boom 22, and corrects a distortion. Specifically, the third correction function discards data indicating an area other than an area determined by a third correction reference (see Fig. 6(A)), the area being an area other than an area surrounded by dot-dash lines in Fig. 6(A), and corrects the distortion. The third correction function receives the captured image data as an argument and outputs the boom upper-surface side surveillance image data as a return value. Note that the boom upper-surface side surveillance image (see Fig. 6(F)) indicated by the boom upper-surface side surveillance image data is configured of an image having a viewpoint directing from the wide-angle camera 18 to the base end of the boom 22 and an image having a viewpoint directing from the wide-angle camera 18 to the tip end of the boom 22. Note that the boom upper-surface side surveillance image may be a single image obtained by combining the image having the viewpoint directing from the wide-angle camera 18 to the base end of the boom 22 and the image having the viewpoint directing from the wide-angle camera 18 to the tip end of the boom 22.
The fourth correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom left-side surveillance image data that captures the left side of the boom 22. Specifically, the fourth correction function is a function that discards data of a portion of the captured image indicated by the captured image data, the portion being an area other than an area portion of the left side of the boom 22 indicated by a left second correction reference (see Fig. 6(A)), in other words, an area other than an area on the left side of a dotted line in Fig. 6(A), and corrects a distortion. The fourth correction function receives the captured image data as an argument and outputs the boom left-side surveillance image data as a return value. The left second correction reference corresponds to "a second correction reference" recited in the claims.
The fifth correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom right-side surveillance image data that captures the right side of the boom 22. Specifically, the fifth correction function is a function that discards data of a portion of the captured image indicated by the captured image data, the portion being an area other than an area portion of the right side of the boom 22 indicated by a right second correction reference (see Fig. 6(A)), in other words, an area other than an area on the right side of the dotted line in Fig. 6(A), and corrects a distortion. The fifth correction function receives the captured image data as an argument and outputs the boom right-side surveillance image data as a return value. The right second correction reference corresponds to "a second correction reference" recited in the claims. The left second correction reference and the right second correction reference correspond to "a second correction reference" recited in the claims.
The third transformation function is a function that rotates an image indicated by image data. The third transformation function receives the image data and a rotation angle as arguments and outputs image data indicating a rotated image as a return value. The third transformation function is an affine transformation (affine matrix) that performs a rotation, for example.
Note that although the areas used to generate the surveillance images are shown in shapes such as a rectangle by the dot-dash lines, the two-dot-dash line, the dotted line, and a broken line in Fig. 6(A), the areas used to generate the surveillance images may have shapes other than the rectangle.
The control program 75 shown in Fig. 1 is a program that controls the driving of the group of hydraulic actuators 27 and the like based on operation signals input from the operating device 29, controls display on the display 81 of the control monitor device 80, accepts inputs of the operator through the control monitor device 80, and executes later-described surveillance image displaying processing (see Fig. 5).
The power supply circuit 74 converts a direct-current voltage supplied from a battery 16 mounted on the traveling body 11 into a direct-current voltage having a predetermined voltage value such as 3.3 V or 5 V, and outputs the converted voltage. The power supply circuit 74 is a DC/DC converter (power supply IC) such as a switching regulator, for example. Note that the battery 16 is charged by the engine 15 of the traveling body 11. The direct-current voltage output by the power supply circuit 74 is supplied to the CPU 71, the operating device 29, the group of sensors 26, the wide-angle camera 18, and the like. Illustrations of power supply lines from the power supply circuit 74 to the group of sensors 26 and the like are omitted in Fig. 1.
Hereinafter, processing executed by the controller 70 of the crane vehicle 10 will be described. Note that the processing executed by the controller 70 is processing that the control program 75 causes the CPU 71 to execute.
First, processing that the controller 70 obtains the second designation information that is the image designation of the operator through the control monitor device 80 and causes the memory 72 to store the information will be described. The controller 70 obtains the first designation information prestored in the memory 72. The controller 70 causes the display 81 to display kinds of images indicated by the first designation information. The operator inputs a kind of image (or kinds of images) that the operator wants to display on the display 84 from among the displayed kinds of images, through the touch sensor 82. For example, if the first designation information is the above-described five kinds of images, choices for the operator are all of the five kinds of images. However, among the five kinds of images, an image definitely displayed on the display 84 without any choice by the operator may be determined in advance. In this case, information indicating the kind of the image definitely displayed on the display 84 is included in the first designation information. Furthermore, the image is excluded from the choices of the operator.
The controller 70 causes the memory 72 to store the second designation information indicating the kind of image (or kinds of images) designated by the operator using the control monitor device 80.
Fig. 5 is a flowchart of the surveillance image displaying processing executed by the controller 70.
The surveillance image displaying processing will be described with reference to Fig. 5. The controller 70 starts the surveillance image displaying processing based on a fact that a power supply of the operating device 29 is turned on or based on a fact that the operator performs a predetermined input to the operating device 29 or the control monitor device 80, for example.
The controller 70 obtains the first designation information and the second designation information stored in the memory 72 (S11). Note that if the operator can select all kinds of images indicated by the first designation information, the controller 70 may obtain only the second designation information indicating the kind of image (or kinds of images) designated (selected) by the operator. Furthermore, if the first designation information includes information indicating the kind of image definitely displayed on the display 84, the controller 70 obtains both the first designation information and the second designation information. Furthermore, if the operator can not designate (select) the image, the controller 70 obtains only the first designation information. In the following, description will be made with collectively describing the first designation information and the second designation information as "designation information". The processing of step S11 corresponds to "designation information obtaining processing" recited in the claims.
The controller 70 obtains the sensor detection values output by each sensor in the group of sensors 26 and causes the memory 72 to store the values (S12). The controller 70 repeatedly obtains the sensor detection values at the predetermined sampling cycle and causes the memory 72 to store the values. Furthermore, the controller 70 obtains the captured image data output by the wide-angle camera 18 (S13).
The controller 70 determines whether the designation information obtained in step S11 indicates display of "rear surveillance image" (S14). Namely, whether the operator has designated (selected) display of "rear surveillance image" or whether the "rear surveillance image" is an image displayed definitely is determined in step S14.
When determining that the designation information indicates the display of "rear surveillance image" (S14: Yes), the controller 70 generates the rear surveillance image data based on the sensor detection value obtained in step S12, the captured image data obtained in step S13, and the first correction function and the first transformation function stored in the memory 72 (S15). To describe in detail, the controller 70 calls the first correction function, passes the captured image data and the derricking angle θ detected by the derricking angle sensor 63 as arguments to the first correction function, and receives the corrected image data as a return value. The controller 70 passes the received corrected image data and the derricking angle θ as arguments to the first transformation function and receives the rear surveillance image data as a return value. An image indicated by the rear surveillance image data is an image that captures a predetermined range around a rear portion of the cabin 13 and is the overhead view image. The processing of step S15 corresponds to "generation processing" and "fourth processing" recited in the claims. The processing of obtaining the corrected image data based on the first correction function, the captured image data, and the derricking angle θ corresponds to "processing of correcting the captured image data based on the fourth correction reference" recited in the claims. The processing of obtaining the rear surveillance image data based on the corrected image data, the derricking angle θ, and the first transformation function corresponds to "transformation processing" recited in the claims.
When determining that the designation information does not indicate the display of "rear surveillance image" (S14: No), the controller 70 skips the processing of step S15.
After executing the processing of step S15, or when determining that the designation information does not indicate the display of "rear surveillance image" (S14: No), the controller 70 determines whether the designation information obtained in step S11 indicates display of "boom tip-end surveillance image" (S16). Namely, whether the operator has designated (selected) the display of "boom tip-end surveillance image" or whether the "boom tip-end surveillance image" is an image displayed definitely is determined in step S16.
When determining that the designation information obtained in step S11 indicates the display of "boom tip-end surveillance image" (S16: Yes), the controller 70 obtains the enlargement ratio based on the boom length indicated by the sensor detection value obtained in step S12 (S17). Specifically, the memory 72 prestores a table that associates the boom length with the enlargement ratio. The controller 70 reads the enlargement ratio associated to the boom length indicated by the sensor detection value in the table.
The controller 70 generates the boom tip-end surveillance image data based on the captured image data obtained in step S13, the enlargement ratio obtained in step S17, and the second correction function and the second transformation function stored in the memory 72 (S18). To describe in detail, the controller 70 calls the second correction function, passes the captured image data to the second correction function as an argument, and obtains pre-enlargement boom tip-end surveillance image data as a return value. Then, the controller 70 calls the second transformation function that performs an enlargement, passes the pre-enlargement boom tip-end surveillance image data and the enlargement ratio to the second transformation function as arguments, and obtains the boom tip-end surveillance image data indicating an enlarged boom tip-end surveillance image as a return value. In other words, the boom tip-end surveillance image that captures the periphery of the tip end portion of the boom 22 is enlarged by the enlargement ratio in accordance with a distance from the wide-angle camera 18 to the tip end portion of the boom 22, and is displayed on the display 84. The processing of step S18 corresponds to "generation processing" and "first processing" recited in the claims. The processing of obtaining the pre-enlargement boom tip-end surveillance image data based on the second correction function and the captured image data in step S18 corresponds to "processing of correcting the captured image data based on the first correction reference" recited in the claims. The processing of obtaining the boom tip-end surveillance image data based on the second transformation function, the pre-enlargement boom tip-end surveillance image data, and the enlargement ratio in step S18 corresponds to "fourth enlargement processing" recited in the claims.
When determining that the designation information obtained in step S11 does not indicate the display of "boom tip-end surveillance image" (S16: No), the controller 70 skips the processing of steps S17 and S18.
After determining in step S16 that the designation information does not indicate the display of "boom tip-end surveillance image", or after executing the processing of step S18, the controller 70 determines whether the designation information obtained in step S11 indicates display of "boom upper-surface side surveillance image" (S19). Namely, whether the operator has designated (selected) the display of "boom upper-surface side surveillance image" or whether the "boom upper-surface side surveillance image" is an image displayed definitely is determined in step S19.
When determining that the designation information obtained in step S11 indicates the display of "boom upper-surface side surveillance image" (S19: Yes), the controller 70 generates the boom upper-surface side surveillance image data based on the captured image data obtained in step S13 and the third correction function stored in the memory 72 (S20). To describe in detail, the controller 70 calls the third correction function, passes the captured image data to the third correction function as an argument, and obtains the boom upper-surface side surveillance image data as a return value. The processing of step S20 corresponds to "generation processing" and "third processing" recited in the claims.
Next, the controller 70 determines whether the boom 22 is in a process of standing up based on the sensor detection values obtained in step S12 (S21). To describe in detail, the controller 70 determines whether the derricking angle θ is increasing over time based on a plurality of derricking angles θ detected by the derricking angle sensor 63 at the predetermined sampling cycle.
When determining that the boom 22 is the process of standing up (S21: Yes), the controller 70 executes first enlargement processing of enlarging the boom upper-surface side surveillance image indicated by the boom upper-surface side surveillance image data generated in step S20 (S22). To describe in detail, the controller 70 calls the second transformation function that performs an enlargement and reads the prestored predetermined enlargement ratio from the memory 72. The controller 70 passes the boom upper-surface side surveillance image data generated in step S20 and the predetermined enlargement ratio to the second transformation function as arguments, and obtains the boom upper-surface side surveillance image data indicating an enlarged boom upper-surface side surveillance image as a return value. The processing of step S22 corresponds to "first enlargement processing" recited in the claims.
When determining that the boom 22 is not in the process of standing up (S21: No), the controller 70 skips the processing of step S22. In other words, when the boom 22 is not in the process of standing up, the boom upper-surface side surveillance image having a prescribed size is displayed on the display 84, and when the boom 22 is in the process of standing up, the boom upper-surface side surveillance image having a size larger than the prescribe size is displayed on the display 84. It is when the boom 22 is in the process of standing up that a possibility that the upper surface 45 of the boom 22 contacts with an obstacle occurs. In other words, the controller 70 enlarges and displays the boom upper-surface side surveillance image when the boom 22 is in the process of standing up and there is the possibility that the upper surface 45 of the boom 22 contacts with the obstacle.
When determining that the designation information obtained in step S11 does not indicate display of "boom upper-surface side surveillance image" (S19: No), the controller 70 skips the processing of steps S20, S21, and S22.
After determining in step S19 that the designation information does not indicate the display of "boom upper-surface side surveillance image", after determining in step S21 that the boom 22 is not in the process of standing up, or after executing the processing of step S22, the controller 70 determines whether the designation information obtained in step S11 indicates display of "boom left-side surveillance image" and "boom right-side surveillance image" (S23). Namely, whether the operator has designated (selected) the display of "boom left-side surveillance image" and "boom right-side surveillance image", or whether the "boom left-side surveillance image" and the "boom right-side surveillance image" are images displayed definitely is determined in step S23.
When determining that the designation information obtained in step S11 indicates the display of "boom left-side surveillance image" and "boom right-side surveillance image" (S23: Yes), the controller 70 determines a first rotation angle and a second rotation angle based on the derricking angle θ of the boom 22 indicated by the sensor detection value obtained in step S12 (S24). For example, the derricking angle θ is determined as the first rotation angle, and -θ is determined as the second rotation angle. The first rotation angle is an angle by which the boom left-side surveillance image is rotated, and the second rotation angle is an angle by which the boom right-side surveillance image is rotated.
The controller 70 generates the boom left-side surveillance image data based on the captured image data obtained in step S13, the fourth correction function and the third transformation function stored in the memory 72, and the first rotation angle determined in step S24 (S25). Furthermore, the controller 70 generates the boom right-side surveillance image data based on the captured image data obtained in step S13, the fifth correction function and the third transformation function stored in the memory 72, and the second rotation angle determined in step S24 (S25). Note that in the following, description may be made with collectively describing the boom left-side surveillance image data and the boom right-side surveillance image data as boom side surveillance image data, or description may be made with collectively describing the boom left-side surveillance image and the boom right-side surveillance image as boom side surveillance images.
The boom left-side surveillance image data corresponds to "boom side surveillance image data" recited in the claims. The boom right-side surveillance image data corresponds to "boom side surveillance image data" recited in the claims. The boom left-side surveillance image data and the boom right-side surveillance image data correspond to "boom side surveillance image data" recited in the claims. The boom left-side surveillance image corresponds to "a boom side surveillance image" recited in the claims. The boom right-side surveillance image corresponds to "a boom side surveillance image" recited in the claims. The boom left-side surveillance image and the boom right-side surveillance image correspond to "a boom side surveillance image" recited in the claims.
The processing of step S25 will be described in detail. The controller 70 calls the fourth correction function, passes the captured image data to the fourth correction function as an argument, and obtains pre-rotation boom left-side surveillance image data as a return value. Next, the controller 70 calls the third transformation function that rotates an image, passes the pre-rotation boom left-side surveillance image data and the first rotation angle to the third transformation function as arguments, and obtains the boom left-side surveillance image data indicating a rotated boom left-side surveillance image as a return value. The ground captured in the rotated boom left-side surveillance image becomes approximately horizontal on the display 84. In other words, the rotation of the image by the third transformation function is performed in order to make the ground captured in the boom left-side surveillance image horizontal on the display 84 so as to remove sense of discomfort felt by the operator. Similarly, the controller 70 calls the fifth correction function, passes the captured image data to the fifth correction function as an argument, and obtains pre-rotation boom right-side surveillance image data as a return value. Next, the controller 70 calls the third transformation function that rotates an image, passes the pre-rotation boom right-side surveillance image data and the second rotation angle to the third transformation function as arguments, and obtains the boom right-side surveillance image data indicating a rotated boom right-side surveillance image as a return value. The ground captured in the rotated boom right-side surveillance image becomes approximately horizontal on the display 84.
The processing of step S25 corresponds to "generation processing" and "second processing" recited in the claims. The processing of obtaining the pre-rotation boom left-side surveillance image data based on the captured image data and the fourth correction function in step S25 corresponds to "processing of correcting the captured image data based on the second correction reference" recited in the claims. The processing of obtaining the pre-rotation boom right-side surveillance image data based on the captured image data and the fifth correction function in step S25 corresponds to "processing of correcting the captured image data based on the second correction reference" recited in the claims. The processing of obtaining the pre-rotation boom left-side surveillance image data and the pre-rotation boom right-side surveillance image data based on the captured image data, the fourth correction function, and the fifth correction function in step S25 corresponds to "processing of correcting the captured image data based on the second correction reference" recited in the claims. The processing of obtaining the boom left-side surveillance image data based on the pre-rotation boom left-side surveillance image data, the first rotation angle, and the third transformation function in step S25 corresponds to "rotation processing" recited in the claims. The processing of obtaining the boom right-side surveillance image data based on the pre-rotation boom right-side surveillance image data, the second rotation angle, and the third transformation function in step S25 corresponds to "rotation processing" recited in the claims. The processing of obtaining the boom left-side surveillance image data and the boom right-side surveillance image data based on the pre-rotation boom left-side surveillance image data, the pre-rotation boom right-side surveillance image data, the first rotation angle, the second rotation angle, and the third transformation function in step S25 corresponds to "rotation processing" recited in the claims.
Next, the controller 70 determines whether the boom 22 is slewing left based on the sensor detection values obtained in step S12 (S26). To describe in detail, the controller 70 determines whether the slewing angle α detected by the slewing angle sensor 61 at the predetermined sampling cycle is increasing over time. If the slewing angle α is increasing, the controller 70 determines that the boom 22 is slewing left.
When determining that the boom 22 is slewing left (S26: Yes), the controller 70 executes second enlargement processing of enlarging the boom left-side surveillance image indicated by the boom left-side surveillance image data generated in step S25 (S27). To describe in detail, the controller 70 calls the second transformation function that performs an enlargement, and reads the prestored predetermined enlargement ratio from the memory 72. The controller 70 passes the boom left-side surveillance image data generated in step S25 and the predetermined enlargement ratio to the second transformation function as arguments, and obtains the boom left-side surveillance image data indicating an enlarged boom left-side surveillance image as a return value. The processing of step S27 corresponds to "second enlargement processing" recited in the claims.
When determining that the boom 22 is not slewing left (S26: No), the controller 70 skips the processing of step S27. In other words, when the boom 22 is not slewing left, the boom left-side surveillance image having a prescribed size is displayed on the display 84, and when the boom 22 is slewing left, the boom left-side surveillance image having a size larger than the prescribed size is displayed on the display 84. It is when the boom 22 is slewing left that a possibility that a left side-surface of the boom 22 contacts with an obstacle occurs. In other words, the controller 70 enlarges and displays the boom left-side surveillance image when the boom 22 is slewing left and there is the possibility that the left side-surface of the boom 22 contacts with the obstacle.
When determining that the boom 22 is not slewing left (S26: No), or after executing the processing of step S27, the controller 70 determines whether the boom 22 is slewing right based on the sensor detection values obtained in step S12 (S28). To describe in detail, the controller 70 determines whether the slewing angle α detected by the slewing angle sensor 61 at the predetermined sampling cycle is decreasing over time. If the slewing angle α is decreasing, the controller 70 determines that the boom 22 is slewing right.
When determining that the boom 22 is slewing right (S28: Yes), the controller 70 executes third enlargement processing of enlarging the boom right-side surveillance image indicated by the boom right-side surveillance image data generated in step S25 (S29). To describe in detail, the controller 70 calls the second transformation function that performs an enlargement, and reads the prestored predetermined enlargement ratio from the memory 72. The controller 70 passes the boom right-side surveillance image data generated in step S25 and the predetermined enlargement ratio to the second transformation function as arguments, and obtains the boom right-side surveillance image data indicating an enlarged boom right-side surveillance image as a return value. The processing of step S29 corresponds to "third enlargement processing" recited in the claims.
When determining that the boom 22 is not slewing right (S28: No), the controller 70 skips the processing of step S29. In other words, when the boom 22 is not slewing right, the boom right-side surveillance image having a prescribed size is displayed on the display 84, and when the boom 22 is slewing right, the boom right-side surveillance image having a size larger than the prescribed size is displayed on the display 84. It is when the boom 22 is slewing right that a possibility that a right side-surface of the boom 22 contacts with an obstacle occurs. In other words, the controller 70 enlarges and displays the boom right-side surveillance image when the boom 22 is slewing right and there is the possibility that the right side-surface of the boom 22 contacts with the obstacle.
When determining that the designation information obtained in step S11 does not indicate the display of "boom side surveillance image" (S23: No), the controller 70 skips the processing from step S24 to S29.
After determining that the designation information does not indicate the display of "boom side surveillance image" (S23: No), after determining as not slewing right (S28: No), or after executing the processing of step S29, the controller 70 inputs the generated image data to the display 84 and cause the display 84 to display the surveillance images (S30). Specifically, the controller 70 inputs the generated surveillance image data to the display 84. The surveillance images displayed on the display 84 are images determined to be displayed according to the specifications (first designation information) and images designated (selected) by the operator. Furthermore, the surveillance images displayed on the display 84 are images enlarged appropriately in accordance with the extension, the retraction, the derricking, and the slew of the boom 22. The processing of step S30 corresponds to "display processing" recited in the claims.
Each surveillance image will be described in detail. The wide-angle camera 18 is attached to the boom 22 and is slewed integrally with the boom 22. Therefore, the rear surveillance image shown in Fig. 6(B) always captures the rear of the cabin 13 (see Fig. 2), not a rear of the crane vehicle 10. In other words, the rear surveillance image always captures the rear of the cabin 13 that becomes a blind spot for the driver. Furthermore, the rear surveillance image is the overhead view image always looking down the rear of the cabin 13 from the same viewpoint, irrespective of the derricking of the boom 22. Furthermore, the rear surveillance image is an image that always captures a constant range, irrespective of the derricking of the boom 22.
The boom left-side surveillance image shown in Fig. 6(C) is an image obtained by looking at the left side from the boom 22, and the boom right-side surveillance image shown in Fig. 6(D) is an image obtained by looking at the right side from the boom 22. The ground captured in the boom left-side surveillance image and the ground captured in the boom right-side surveillance image are approximately horizontal on the display 84, and obstacles standing vertically from the ground in the boom left-side surveillance image and the boom right-side surveillance image extend approximately along the up-down direction on the display 84.
The boom tip-end surveillance image shown in Fig. 6(E) captures the surroundings of the tip end portion of the boom 22. A range captured in the boom tip-end surveillance image is determined to be a range that allows the operator who notices an obstacle in the boom tip-end surveillance image to stop the crane apparatus 12 before the tip end portion of the boom 22 contacts with the obstacle, for example. The range is constant irrespective of the extension length of the boom 22 by the enlargement (S17, S18) of the boom tip-end surveillance image.
The boom upper-surface side surveillance image shown in Fig. 6(F) consists of an image of the tip end of the boom 22 and its upside viewed from the wide-angle camera 18, and an image of the base end of the boom 22 and its upside viewed from the wide-angle camera 18. In other words, the boom upper-surface side surveillance image is an image that captures the area facing the upper surface 45 of the boom 22. Note that the two images may be displayed side by side in a left-right direction on the display 84, or may be displayed side by side in the up-down direction on the display 84. Or, the two images may be combined and displayed as a single image on the display 84.
The five surveillance images shown in Figs. 6(B) to 6(F) may be arranged and displayed in any manner on the display 84.
Furthermore, the boom side surveillance image and the boom upper-surface side surveillance image that are enlarged may be displayed on the display 84 separately from the boom side surveillance image and the boom upper-surface side surveillance image before enlargement. For example, when the boom 22 is not slewed, the boom side surveillance image before enlargement is displayed on the display 84, and when the boom 22 is slewed, the boom side surveillance image after enlargement is displayed on the display 84 with the boom side surveillance image before enlargement.
As shown in Fig. 5, after executing the processing of step S30, the controller 70 determines whether to finish the surveillance image displaying processing (S31). The controller 70 determines to finish the surveillance image displaying processing, based on a fact that the power supply of the operating device 29 is turned off or based on a fact that a finish instruction is input through the control monitor device 80, for example (S31: Yes, End). When determining not to finish the surveillance image displaying processing (S31: No), the controller 70 executes the processing of and after S11 again. In other words, the controller 70 repeatedly executes the surveillance image displaying processing. A cycle (sampling cycle) at which the controller 70 repeats the surveillance image displaying processing is set between several hundred milliseconds and several minutes, for example.

[Actions and effects of embodiment]

Since the wide-angle camera 18 is attached to the upper surface 45 of the boom 22, the camera can capture the area around the tip end portion of the boom 22, left and right side areas of the boom 22, the area facing the upper surface 45 of the boom 22, and the rear area being surroundings of the base end of the boom 22 together. The controller 70 generates the surveillance image data from the captured image data with respect to the area indicated by the first designation information prestored in the memory 72 or the area designated (selected) by the operator, and causes the display 84 to display the surveillance image indicated by the generated surveillance image data. Therefore, even if an area that becomes the blind spot or a surveillance area requested by the operator of the crane vehicle 10 is the area around the tip end of the boom 22, the side areas of the boom 22, the area facing the upper surface 45 of the boom 22, or the rear area around the base end of the boom 22 (rear area of the cabin 13), the safety device 17 can cause the display 84 to display the surveillance image that captures the area. Therefore, the crane apparatus 12 can eliminate the blind spot irrespective of the type or the specifications, or can eliminate the blind spot in accordance with a request of the operator.
The safety device 17 configured of the wide-angle camera 18, the display 84, and the controller 70 can change the kind of image displayed on the display 84 by the first designation information stored in the memory 72. Therefore, the safety device 17 can be commonly used for various crane apparatuses 12. As a result, sharing of parts can be attempted.
The controller 70 reads the second designation information indicating the image designation of the operator from the memory 72, and causes the display 84 to display the surveillance image of the kind designated by the second designation information. Therefore, the surveillance image wanted by the operator can be displayed on the display 84.
The position and the height of the wide-angle camera 18 taking the slewing base 21 as a reference are changed by the derricking of the boom 22. The processing of transforming the viewpoint for obtaining the overhead view image is performed based on the derricking angle of the boom 22 that is a parameter that determines the position and the height of the wide-angle camera 18. Therefore, the crane apparatus 12 can cause the display 84 to display the rear surveillance image always overlooking the rear of the cabin 13 from a fixed position, not depending on the derricking angle of the boom 22.
The boom side surveillance image is tilted in accordance with the derricking of the boom 22. The controller 70 rotates the boom side surveillance image in accordance with the derricking angle of the boom 22 to rectify a tilt of the boom side surveillance image. Therefore, the crane apparatus 12 can allow the operator to easily recognize the obstacle compared to a case where the tilt of the boom side surveillance image is not rectified.
When the boom 22 is in the process of standing up, there occurs a risk of a contact between the upper surface 45 of the boom 22 and an obstacle. When the boom 22 is in the process of standing up, the controller 70 enlarges and displays the boom upper-surface side surveillance image. Therefore, the crane apparatus 12 allows the operator to easily recognize the obstacle compared to a case where the boom upper-surface side surveillance image is always displayed at a fixed size on the display 84.
When the boom 22 is slewed, there occurs a risk of a contact between a side surface of the boom 22 and an obstacle. The controller 70 enlarges and displays the boom left-side surveillance image when the boom 22 is slewed left, and enlarges and displays the boom right-side surveillance image when the boom 22 is slewed right. Therefore, the crane apparatus 12 can cause the operator to easily recognize the obstacle compared to a case where the boom side surveillance image is always displayed on the display 84 at a fixed size.
Since the wide-angle camera 18 is provided to the base boom 42 whose position is not changed by the extension or the retraction of the boom 22, a wired connection between the wide-angle camera 18 and the controller 70 becomes easier compared to a case where the wide-angle camera 18 is provided to the intermediate boom 43 or the top boom 44 whose position is changed by the extension or the retraction of the boom 22.
In a configuration where the wide-angle camera 18 is provided to the base boom 42, the tip end of the boom 22 leaves the wide-angle camera 18 as the boom 22 extends. Since the boom tip-end surveillance image is enlarged by the enlargement ratio in accordance with the extension length of the boom 22, the crane apparatus 12 can allow the operator to easily recognize an obstacle approaching the tip end of the boom 22.

[Modification examples]

In the above-described embodiment, described is an example in which the angle of view of the wide-angle camera 18 is 220 degrees. However, the angle of view of the wide-angle camera 18 may be set to another angle of view between 180 degrees and 220 degrees, such as 180 degrees or 200 degrees.
In the above-described embodiment, described is an example in which the crane apparatus 12 is of a mobile type (crane vehicle 10). However, the crane apparatus 12 may be of a fixed type, such as a tower crane.
In the above-described embodiment, described is an example in which the first designation information is prestored in the memory 72 and a second designation value input by the operator is stored in the memory 72. In other words, described is an example in which the surveillance image to be displayed is determined in advance depending on the type or the specifications, and the operator can select the surveillance image to be displayed. However, the first designation information may not necessarily be prestored in the memory 72, or a function that the operator inputs the second designation value may not necessarily be provided. In other words, the operator may only be able to select the surveillance image to be displayed, or the surveillance image to be displayed may be determined in advance by the type or the specifications, and the operator may be unable to select the surveillance image.
In the above-described embodiment, described is an example in which the boom left-side surveillance image is enlarged when the boom 22 is slewed left, and the boom right-side surveillance image is enlarged when the boom 22 is slewed right. However, the boom left-side surveillance image and the boom right-side surveillance image may not necessarily be displayed on the display 84 when the boom 22 is not slewed, the boom left-side surveillance image may be displayed in a normal manner or the enlarged manner when the boom 22 is slewed left, and the boom right-side surveillance image may be displayed in the normal manner or the enlarged manner when the boom 22 is slewed right.
In the above-described embodiment, described is an example in which the boom upper-surface side surveillance image is enlarged when the boom 22 is being caused to stand up. However, the boom upper-surface side surveillance image may not necessarily be displayed on the display 84 when the boom 22 is being caused to stand up, and the boom upper-surface side surveillance image may be displayed in the normal manner or the enlarged manner when the boom 22 is being caused to stand up.
In the above-described embodiment, described is an example in which the first correction function that corrects the captured image and the first transformation function that changes the viewpoint are separate functions. However, the first correction function and the first transformation function may be a single function. Similarly, the second correction function and the second transformation function may be single function, the fourth correction function and the third transformation function may be a single function, or the fifth correction function and the third transformation function may be a single function.
In the above-described embodiment, described is an example in which corrections and transformations of image data are performed using functions. However, a class that generates the surveillance image data as an instance (object), an arithmetic formula, or the like may be used, as long as the corrections and the transformations of the image data can be performed.

DESCRIPTION OF REFERENCE CHARACTERS

10: crane vehicle
11: traveling body
12: crane apparatus
13: cabin
17: safety device
18: wide-angle camera
19: cable
21: slewing base
22: boom
23: winch
26: group of sensors
29: operating device
41: wire rope
42: base boom (tube body)
43: intermediate boom (tube body)
44: top boom (tube body)
45: upper surface
51: slewing motor
52: derricking cylinder
53: extending/retracting cylinder
54: hydraulic motor
61: slewing angle sensor
62: boom length sensor
63: derricking angle sensor
70: controller
71: CPU
72: memory
75: control program
80: control monitor device
81: display
82: touch sensor
84: display

Claims

A crane apparatus comprising:
a slewing base;

a cabin mounted on the slewing base and in which an operating device is installed;

a boom provided to the slewing base so as to be able to stand and lie, the boom being able to extend and retract; and

a safety device having a wide-angle camera attached to an upper surface of the boom in a lie-down posture, a controller, and a display, wherein

the controller is configured to execute:
generation processing of generating surveillance image data based on captured image data generated by the wide-angle camera; and

display processing of causing the display to display a surveillance image indicated by the surveillance image data.
The crane apparatus according to claim 1, wherein
the controller is configured to further execute designation information obtaining processing of obtaining designation information, and

the generation processing is one or more processes indicated by the designation information and selected from among:
first processing of generating boom tip-end surveillance image data by correcting the captured image data based on a first correction reference in accordance with a tip end of the boom captured in a captured image indicated by the captured image data;

second processing of generating boom side surveillance image data by correcting the captured image data based on a predetermined second correction reference in an area captured in the captured image indicated by the captured image data, the area being on a side of the boom;

third processing of generating boom upper-surface side surveillance image data by correcting the captured image data based on a predetermined third correction reference in an area captured in the captured image indicated by the captured image data, the area facing the upper surface of the boom; and

fourth processing of generating rear surveillance image data by correcting the captured image data based on a fourth correction reference in accordance with a base end of the boom captured in the captured image indicated by the captured image data.
The crane apparatus according to claim 2, further comprising a derricking angle sensor configured to output a first detection value indicating a derricking angle of the boom, wherein
the fourth processing includes:
processing of correcting the captured image data based on the fourth correction reference; and

transformation processing of transforming a viewpoint based on the derricking angle indicated by the first detection value, and

a rear surveillance image indicated by the rear surveillance image data is an overhead view image indicating a rear of the cabin.
The crane apparatus according to claim 2, further comprising a derricking angle sensor configured to output a first detection value indicating a derricking angle of the boom, wherein
the second processing further includes:
processing of correcting the captured image data based on the second correction reference; and

rotation processing of rotating a boom side surveillance image that is a surveillance image indicated by the boom side surveillance image data, by a rotation angle in accordance with the derricking angle indicated by the first detection value.
The crane apparatus according to claim 2, further comprising a derricking angle sensor configured to output a first detection value indicating a derricking angle of the boom, wherein
the controller is configured to:
execute the third processing based on a fact that the designation information indicates an execution of the third processing; and

execute first enlargement processing of enlarging a boom upper-surface side surveillance image indicated by the boom upper-surface side surveillance image data generated by the third processing, based on a fact that the boom is determined to be in a process of standing up by a change over time of the derricking angle indicated by the first detection value, before executing the display processing.
The crane apparatus according to claim 2, further comprising a slewing angle sensor configured to output a second detection value indicating a slewing angle of the slewing base, wherein
the boom side surveillance image data includes boom left-side surveillance image data indicating a left side of the boom and boom right-side surveillance image data indicating a right side of the boom, and

the controller is configured to:
execute the second processing based on a fact that the designation information indicates an execution of the second processing;

execute second enlargement processing of enlarging a boom left-side surveillance image indicated by the boom left-side surveillance image data generated by the second processing, based on a fact that a left slew is determined by the second detection value, before executing the display processing; and

execute third enlargement processing of enlarging a boom right-side surveillance image indicated by the boom right-side surveillance image data generated by the second processing, based on a fact that a right slew is determined by the second detection value, before executing the display processing.
The crane apparatus according to claim 1, wherein
the boom has a plurality of tube bodies including a base boom, and

the wide-angle camera is provided to the base boom.
The crane apparatus according to claim 2, further comprising a boom length sensor configured to output a third detection value indicating an extension length of the boom, wherein
the boom has a plurality of tube bodies including a base boom,

the wide-angle camera is provided to the base boom, and

the first processing includes:
processing of correcting the captured image data based on the first correction reference; and

fourth enlargement processing of enlarging by an enlargement ratio in accordance with the third detection value.
The crane apparatus according to claim 1, wherein an angle of view of the wide-angle camera is equal to or larger than 180 degrees.