JP2014105091A - Monitoring camera of crane - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a crane in which it is possible to obtain a large base line length defined as distance between optical axes of a pair of monitoring cameras when two monocular monitoring cameras are provided to configure a stereo camera and image processing is performed by capturing an image with the stereo camera.SOLUTION: The crane has an operation monitoring device in which two monocular monitoring cameras 2A and 2B are provided for configuring a stereo camera to capture an image and which performs image processing. When a direction orthogonal to an extending direction of a boom that can extend and tilt is a width direction of the boom, two monocular monitoring cameras 2A and 2B in which distance between optical axes of the cameras is defined as a base line length D are provided at both ends in the width direction of the head 17d of the boom.

Description

  The present invention relates to an improvement in a crane provided with a work monitoring device in which a monitoring camera is provided at the tip of an extendable boom, image processing is performed on an image captured by the monitoring camera, and an image is displayed on a monitor device.

  Conventionally, a working machine such as a crane provided with a work monitoring device for monitoring a work site by providing a monitoring camera on an extendable boom head and displaying a photographed image taken by the monitoring camera on the screen of the monitor device. Is known (see Patent Document 1).

  In addition, there is also known a technique in which a stereo camera is attached to an arm of a work machine such as a crusher, and a three-dimensional shape of a target object at a work site is obtained and displayed on a screen of a monitor device (see Patent Document 2).

JP 2011-151742 A JP 2010-248777 A

By the way, when working with a work machine such as a crane, when there is one surveillance camera, a blind spot portion that is a shadow of a suspended load or a hook block is generated. Therefore, it is conceivable that two surveillance cameras are provided and a stereo camera is constructed and imaged, and image processing is performed to interpolate each other as a blind spot.

  In that case, it is desirable to configure so that the accuracy of image processing can be increased as much as possible by increasing the base line length of the surveillance camera.

  It is also desirable to be able to accurately measure the height of a target object such as a building from the ground using an image captured by a surveillance camera.

  The present invention has been made in view of the above circumstances. In the case where two monitoring cameras are provided and a stereo camera is constructed to perform imaging and image processing is performed, the distance between the optical axes of the pair of monitoring cameras is determined. An object of the present invention is to provide a crane capable of improving the image processing accuracy by increasing the defined base line length.

The crane according to the present invention is a crane having a work monitoring device that performs image processing by constructing a stereo camera by providing two monocular monitoring cameras and performing image processing, at least on both sides of the tilting surface of the tiltable boom. A monocular surveillance camera in which the distance between the optical axes is defined as a base line length is provided on each side of the boom in the width direction as a boom width direction.

  According to the present invention, the monocular surveillance camera in which the distance between the optical axes is defined as the base line length is provided on both sides in the width direction of the tip of the boom with at least both sides of the tilting surface of the boom that can tilt as the width direction of the boom. When a stereo camera is constructed using a monocular monitoring camera, the base line length can be increased, and as a result, the image processing accuracy can be improved.

FIG. 1 is an explanatory diagram of a crane to which a stereo camera is attached. FIG. 2 is a schematic diagram showing a mounting state of the stereo camera shown in FIG. FIG. 3 is a block diagram of the arithmetic processing circuit according to the embodiment of the present invention. 4A is an explanatory diagram illustrating an example of an image displayed on the screen of the monitor unit illustrated in FIG. 3, and is disposed on a side far from the cab with reference to the direction in which the boom of the crane illustrated in FIG. 1 extends. It is a figure which shows the image imaged with the surveillance camera which is. 4B is an explanatory diagram illustrating an example of an image displayed on the screen of the monitor unit illustrated in FIG. 3, and is based on a monitoring camera close to the cab with respect to the direction in which the boom of the crane illustrated in FIG. 1 extends. It is a figure which shows the imaged image. FIG. 4C shows a combined image of the image shown in FIG. 4A and the image shown in FIG. 4B. FIG. 5 is an explanatory diagram for explaining the principle of distance measurement to a subject to be photographed by a stereo camera.

(Description of crane to which the present invention is applied)
Hereinafter, an embodiment of the crane of this stereo camera will be described with reference to the drawings.
In FIG. 1, reference numeral 11 denotes a crane. This crane 11 is schematically configured from a traveling vehicle body (carrier) 12, a swivel 13 that can be swiveled in a horizontal direction, an operator cab 14, an outrigger 15 that fixes the carrier to the ground (ground) S, and a bracket 16. Has been.

The bracket 16 is provided with a boom 17. The base end portion of the boom 17 is rotatably attached via a support shaft 18 and is raised and lowered by a raising and lowering cylinder 19.
The boom 17 includes, for example, a telescopic base end boom 17a, an intermediate boom 17b, and a front end boom 17c.

  The boom 17 is connected to each expansion / contraction cylinder (not shown) and is expanded and contracted by each expansion / contraction cylinder. The tip boom 17c is provided with a boom head (tip portion) 17d as shown in an enlarged view in FIG. The boom head 17d is provided with a sheave shaft 17f, and the sheave shaft 17f is provided with a pair of sheaves 17e.

  The bracket 16 shown in FIG. 1 is provided with a winch (not shown), and a wire W extending from the winch is stretched over the sheave 17e. A hook block 20 is suspended from the wire W, and a hook 21 is provided on the hook block 20.

  The boom head 17d is provided with two monocular surveillance cameras (CCD cameras) 2A and 2B. The monitoring cameras 2A and 2B are used to construct a stereo camera and constitute a part of a work monitoring apparatus that performs image processing.

Here, the two surveillance cameras 2A and 2B are attached to the sheave shaft 17f. The surveillance cameras 2A and 2B are arranged at symmetrical positions with respect to the direction in which the boom 17 extends with respect to the width direction of the boom 17, but in the case of the boom 17 that can only be tilted, The surveillance cameras 2A and 2B are arranged at symmetrical positions with both sides of the tilt locus plane (tilt plane) drawn by the boom 17 as the width direction.
The distance between the optical axes of the monitoring cameras 2A and 2B is defined as a base line length D, and this base line length D is stored and saved in a memory of an image processing unit to be described later.

Here, each of the surveillance cameras 2A, 2B is provided with a rotation mechanism (not shown) so as to be directed in the vertical direction by its own weight. The surveillance cameras 2A and 2B may be provided with a pan / tilt mechanism for rotating in the pan direction and the tilt direction, and a zoom function for changing the photographing magnification. However, detailed description thereof is omitted here. To do.

  The cab 14 is provided with an operation panel (not shown), an operation lever, an arithmetic processing section (to be described later), and a monitor section (display section), and the operator turns the boom 17 by appropriately operating the operation lever and the operation panel. Undulation, extension, outrigger 15 overhanging, storage, engine start, stop, monitor on / off, etc.

(Explanation of arithmetic processing unit)
FIG. 3 is a block diagram of the arithmetic processing unit according to the embodiment of the present invention.
In FIG. 3, reference numeral 22 indicates an arithmetic processing unit. This arithmetic processing unit 22 receives image signals from the monitoring cameras 2A and 2B, and sensor signals from the cylinder pressure sensor 23, the boom length sensor 24, the undulation angle sensor 25, and the outrigger extension sensor 26.

  The arithmetic processing unit 22 includes an image processing unit 22A and a control processing unit 22B. The control processing unit 22B performs a control process corresponding to the sensor signal from each sensor, and if necessary. Then, information necessary for control is output to the monitor unit 27 as appropriate.

  Imaging signals are input to the image processing unit 22A from the monitoring cameras 2A and 2B. The image processing unit 22A appropriately performs image processing on the imaging signals from the monitoring cameras 2A and 2B and outputs a display signal to the monitoring unit 27. To do. As a result, the images shown in FIGS. 4A to 4C are displayed on the monitor unit 27.

  Here, on the screen G of the monitor unit 27, the image W ′ of the wire W, the image 20 ′ of the hook block 20, the image 12 ′ of the front portion of the vehicle body 12 (see FIG. 4C), the image 15 ′ of the outrigger 15 ( 4C), an image Ref ′ (see FIG. 4C) of a reference target Ref, which will be described later, and an image Ob ′ of the target object Ob.

Here, the arithmetic processing unit 22 is provided with a screen changeover switch SW1 and an image composition processing switch SW2. On the screen G of the monitor unit 27, as shown in FIG. 4B, the monitoring camera 2B disposed on the side close to the cab 14 (the monitoring camera 2B disposed on the right side when viewed from the operator of the driver's seat) is used. The captured image is preferentially displayed.
In the initial state, it is possible to display a scene similar to that seen from the driver's seat side, so it is easy to grasp the surrounding situation.

When the operator operates the screen changeover switch SW1, the image processing unit 22A operates as shown in FIG. 4A from the image captured by the monitoring camera 2B disposed on the side close to the cab 14 shown in FIG. 4B. Control is performed so that the screen display is switched to an image captured by the monitoring camera 2A disposed on the side far from the room 14 (the monitoring camera 2A disposed on the left side when viewed from the operator of the cab 14).

When the operator operates the image composition processing switch SW2, the image processing unit 22A displays a composition processing image of the image captured by the monitoring camera 2B and the image captured by the monitoring camera 2A as shown in FIG. 4C. As described above, image processing is executed.
In FIG. 4C, since the composite processing image is displayed with the hook 21 as the center of the screen G, it is convenient when carrying out the lifting work.

Here, when constructing a composite image, the image processing unit 22A performs image processing so that the image 20 ′ of the hook block 20 is positioned at the center O ′ in the left-right direction of the screen G, and composites it with the screen G. Display the processed image.
In this embodiment, the screen changeover switch SW1 is described as being provided in the arithmetic processing unit 22. However, the screen changeover switch SW1 may be provided in the monitor unit 27 as indicated by a broken line SW3.

  According to this embodiment, when a blind spot is generated in an image captured by one of the monitoring cameras 2A and 2B, the display of the screen G is switched to the image captured by either one. Alternatively, by displaying the composite processing image on the screen G, the work can be performed with as few blind spots as possible. It should be noted that when displaying the composite processed image on the screen G, the image 20 ′ of the hook block 20 is erased so that the image of the part that becomes a blind spot by the hook block 20 and the image 20 ′ of the hook block 20 are displayed. This is desirable from the viewpoint of eliminating the sense of incongruity of the composite processed images that appear to overlap.

The image 20 ′ of the hook block 20 is erased by, for example, acquiring an image of a part that becomes a blind spot by the hook block 20 when the hook block 20 is moved away from the blind spot during the movement of the hook block 20, and storing it in the memory. At the same time, an image 20 ′ of the hook block 20 is erased at the same time as synthesizing an image of a part that becomes a blind spot.
In addition, it can also be set as the structure which displays preferentially the image of the part which becomes a blind spot with the hook block 20, and can also make the image 20 'of the hook block 20 into a transmission image.

  Furthermore, since the surveillance camera 2A, 2B can be provided only on one side of the boom head 17d and the base line length D can be increased as compared with the case where a stereo camera is constructed, the image processing accuracy can be improved, and the height measurement accuracy described later. Improvements can be made.

(Description of height measurement principle)
FIG. 5 shows a measurement principle for measuring a distance to an imaging target using a stereo camera.
The monitoring cameras 2A and 2B have imaging lenses 3A and 3B and two-dimensional imaging elements 4A and 4B, respectively. Symbols O1 and O2 indicate the optical axes of the imaging lenses 3A and 3B, respectively. The focal length of the imaging lenses 3A and 3B is f, the distance from the subject 1 to the monitoring cameras 2A and 2B is L, and the distance L is very large with respect to the focal length f of the monitoring cameras 2A and 2B. To do.

  When the subject 1 is photographed by the surveillance cameras 2A and 2B, the luminous flux from the subject 1 is incident on the imaging lenses 3A and 3B as indicated by arrows P1 and P2, respectively, and the subject image is captured on the two-dimensional imaging elements 4A and 4B. Images are formed as 1A and 1B. Since the imaging positions of the imaging target images 1A and 1B with respect to the two-dimensional imaging elements 4A and 4B are different, this causes a parallax Δ.

By obtaining this parallax Δ, the distance L to the photographing object 1 is obtained using an arithmetic expression L = D × (f / Δ) using the principle of triangulation.
The parallax Δ is obtained by obtaining the pixel positions of the photographing target images 1A and 1B with respect to the pixels (pixel centers) on the optical axes O1 and O2 for each of the monitoring cameras 2A and 2B. Here, as the baseline length D increases, the parallax Δ also increases, so that the distance measurement accuracy can be improved even if the number of pixels of the image sensors 4A and 4B is constant.

  In the above description, the case where the subject 1 is present at a symmetrical position with respect to the monitoring cameras 2A and 2B has been described. However, as indicated by reference numeral 1 ′, the subject 1 ′ is asymmetric with respect to the surveillance cameras 2A and 2B. Even in the case of the position, the arithmetic expression is complicated, but the distance to the photographing object 1 ′ can be obtained using the same principle.

Here, if the heights of the surveillance cameras 2A and 2B are geometrically determined from the postures of the crane and the boom, the height to the subject to be photographed is subtracted from the height direction distance L from the height to the subject 1 '. Can be requested. Further, if the crane is provided with the reference object Ref, the height can be obtained regardless of geometric calculation.
This will be described below.
The image processing unit 22A has a function of determining whether or not both the image Ref ′ of the reference target Ref and the image Ob ′ of the target object Ob are captured in the captured images captured by the monitoring cameras 2A and 2B.

    Here, the reference object Ref is formed as a cross mark on the front surface of the vehicle body 12 as shown in FIG. The image processing unit 22A acquires whether or not the image Ref ′ is present at any pixel position of the two-dimensional imaging devices 4A and 4B, for example, by a known pattern matching method from the captured images acquired by the monitoring cameras 2A and 2B. To do.

  Similarly, the image processing unit 22A wants to measure the height at any pixel position of the two-dimensional imaging devices 4A and 4B from the captured images acquired by the monitoring cameras 2A and 2B, for example, by a known pattern matching method. Whether or not an image Ob ′ of Ob exists is acquired.

  When there are a plurality of target objects Ob, the target object Ob whose height is to be measured is designated. When the monitor unit 27 is a touch panel type, the target object Ob can be specified by the operator pointing on the screen G each time. If the monitor unit 27 is not a touch panel type, for example, the outline of the target object Ob can be automatically extracted and a plurality of target objects Ob can be designated.

Here, as illustrated in FIG. 1, the calculation unit 22C uses a captured image including a reference object Ref indicating the reference height Zref from the ground S, and uses each pixel of the image Ref ′ of the reference object Ref. The parallax Δ is obtained from the position, and the height direction distance Z ₁ from the monitoring cameras 2A, 2B to the reference object Ref is calculated using the parallax Δ.

Similarly, the calculation unit 22C obtains a parallax Δ from each pixel position of the image Ob ′ of the target object Ob whose height is to be measured, and uses the parallax Δ in the height direction from the monitoring cameras 2A and 2B to the reference target Ref. calculating the distance _{Z 2.}
Then, the height Z of the target object Ob from the ground S is obtained by the following formula using the height direction distances Z ₁ and Z ₂ thus obtained and the reference height Zref from the ground S.
Z = (Z ₁ + Zref) −Z ₂

That is, the calculation unit 22C calculates the height direction distance Z ₁ from the monitoring cameras 2A and 2B to the reference object Ref using an image including the reference object Ref indicating the reference height Zref from the ground S. The height direction distance Z ₂ from the monitoring cameras 2A, 2B to the target object Ob is calculated using the image including the target object Ob, and the height direction distance Z ₁ and the reference height Zref to the reference target Ref are calculated. The height Z of the target object Ob from the ground S is calculated by subtracting the height direction distance Z ₂ to the target object Ob from the sum of.

  According to this embodiment, since the reference object Ref indicating the height reference from the ground S is provided in the crane 11, even when the height of the target object Ob from the ground S is measured by the monitoring cameras 2A and 2B, it is accurate. In addition, the height of the target object Ob with respect to the ground S can be accurately measured.

As mentioned above, although the Example was described, this invention is not restricted to this, The following are included.
(1) In the embodiment, since the surveillance cameras 2A and 2B are provided on the sheave shaft 17f, it is possible to attach them without significantly changing existing components.
However, it is also possible to provide a mounting shaft portion for mounting the monitoring cameras 2A and 2B in parallel with the sheave shaft 17f and to provide the monitoring cameras 2A and 2B on the mounting shaft portion. With this configuration, it is possible to prevent the vibration of the sheave shaft 17f from being transmitted to the monitoring cameras 2A and 2B.

(2) In the embodiment, the monitoring cameras 2A and 2B are provided on the boom head (tip portion) 17d of the boom 17. However, in order to prevent the boom 17 from being laterally bent, the boom 17 is horizontal in the width direction. The monitoring cameras 2A and 2B may be provided on a protruding attachment member (not shown) that extends so as to extend in the direction. If comprised in this way, base line length can be taken much larger compared with the case where it provides in the sheave shaft 17f.

(3) Once the crane 11 is installed at the work site, the crane 11 is not moved until the work is completed, so images acquired with the movement of the boom 17 are sequentially stored in a memory, and the imaging range changes with the movement of the boom 17. Even in such a case, if the image acquired before the imaging range change can be used, the image processing speed can be improved.

(4) Since the target object Ob appears smaller as the height of the monitoring cameras 2A and 2B increases, a precise image can be obtained by increasing the pixel density of the processed image.
(5) In order to grasp the position of the target object Ob, the position of the feature point such as the corner of the target object Ob is obtained a plurality of times, and the average value is acquired, thereby improving the position acquisition accuracy of the target object Ob. be able to.

(6) In the embodiment, an image viewed from directly above the hook 21 is displayed on the screen G as a composite processing image. However, the composite processing image is not limited to this, and various bird's-eye views that have undergone viewpoint conversion are used. An image can be created and displayed. An overhead image conversion switch can be provided in the image processing unit, and an overhead image can be appropriately displayed on the screen G by an operator's operation.

2A, 2B ... surveillance camera 11 ... crane 14 ... cab 17 ... boom 27 ... monitor unit 22A ... image processing unit

Claims (6)

A crane having a work monitoring device that provides two monocular surveillance cameras, constructs a stereo camera, performs imaging, and performs image processing,
A monocular surveillance camera in which the distance between the optical axes is defined as the base line length is provided on each side of the width direction of the tip of the boom with at least both sides of the tilting surface of the boom that can be tilted as the width direction of the boom. Features crane.   An image processing unit that inputs an image pickup signal from each of the monitoring cameras and performs image processing and outputs the image signal to the monitor unit, and switches an image picked up by the monitoring camera to a screen of the monitor unit The crane according to claim 1, wherein the crane is displayable.   3. The crane according to claim 2, wherein an image captured by a monitoring camera disposed on a side close to a cab is preferentially displayed on the screen of the monitor unit.   The image processing unit synthesizes an image captured by one monitoring camera and an image captured by the other monitoring camera, and a combined processing image synthesized by the image processing unit is displayed on the screen of the monitor unit. The crane according to claim 1, wherein the crane is displayed.   5. A sheave shaft for supporting a sheave on which a rope for hanging a hook is hung by each surveillance camera, and is provided at a symmetrical position with respect to the boom. The crane according to item. 2. The monitoring camera according to claim 1, wherein each of the monitoring cameras is provided on an overhanging attachment member that projects from the boom so as to extend in the horizontal direction in the width direction thereof in order to prevent lateral deformation of the boom. crane.