JP2014062803A - Monitoring camera device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a monitoring camera device which performs stereoscopic photography with one monocular camera, while calculating a height of an object imaged.SOLUTION: A monitoring camera device including a monocular monitoring camera 100 attached near a tip of a telescopic boom of a crane, includes an image processing controller which acquires at least first and second images captured by the monitoring camera 100, while the telescopic boom is moved from a storage position to an operation start position, to calculate a height of an object, on the basis of an optical axis distance between a first imaging position of the first image and a second imaging position of the second image and a position of the object in the first and second images.

Description

  The present invention relates to a surveillance camera device that acquires a stereoscopic image from an image captured by a monocular camera attached near the tip of a boom of a crane and obtains the height of an object such as a captured structure.

  2. Description of the Related Art Conventionally, there has been known a stereo camera device in which a stereo camera is provided in front of a boom of a work machine so that a work object in front of the work machine is stereoscopically photographed (see Patent Document 1).

  The work machine includes a lower traveling body, an upper swing body provided on the lower travel body, a boom that can be raised and lowered on the upper swing body, a sheave provided on a tip portion of the boom, and a tip of the sheave And a pair of stereo cameras provided on the front surface of the boom.

  The pair of stereo cameras are directed forward, and stereoscopically captures the work object to be crushed in front of the work machine, and the depth in the front-rear direction of the work object is obtained by stereoscopic shooting.

JP 2010-248777 A

  In such a working machine, there is a problem that a pair of stereo cameras for three-dimensional imaging is necessary to obtain the depth of the work object.

  An object of the present invention is to provide a surveillance camera device capable of obtaining the height of an object by stereoscopic shooting with a single monocular camera.

The invention of claim 1 is a surveillance camera device comprising a monocular camera attached in the vicinity of the tip of a boom of a crane,
While the boom is moved from the storage position, an image captured by the monocular camera is captured, and the two first and second images captured and the distance between the optical axes of the first and second images are captured. An image calculation processing device for obtaining the height of an object being imaged is provided.

  According to the present invention, stereoscopic imaging can be performed with a single monocular camera, and the height of the captured structure can be accurately obtained.

It is the side view which showed the mobile crane carrying the monitoring camera apparatus which concerns on this invention. It is the block diagram which showed the structure of the monitoring camera apparatus. It is the block diagram which showed the structure of the image processing controller of the monitoring camera apparatus. It is explanatory drawing which shows that the position which a monitoring camera images according to expansion | extension of an expansion-contraction boom changes. (A) is explanatory drawing which shows the image imaged in the 1st position shown in FIG. 4, (B) is explanatory drawing which shows the image imaged in the 2nd position shown in FIG. It is explanatory drawing which showed the positional relationship and coordinate of a camera on either side. It is explanatory drawing which shows the relationship between the point on the three-dimensional space, and the position on the image of the left and right cameras which imaged this point. It is explanatory drawing which shows that the position of the point of three-dimensional space can be calculated | required. It is explanatory drawing which shows an example of the image which synthesize | combined and displayed the mark M which shows the position of the extracted feature point, and the height of the calculated | required structure.

  Hereinafter, an embodiment which is an embodiment of a surveillance camera device according to the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 shows a rough terrain crane 10 as a crane equipped with a monitoring camera device. The rough terrain crane 10 includes a carrier 11 serving as a main body portion of a vehicle having a traveling function, a pair of left and right front outriggers 12 provided on the front side of the carrier 11, and a pair of left and right sides provided on the rear side of the carrier 11. A rear outrigger 13, a swivel base 14 attached to the upper part of the carrier 11 so as to be horizontally swivelable, a cabin 20 provided on the swivel base 14, a telescopic boom 16 attached to a bracket 15 fixed to the swivel base 14, etc. And.

  The telescopic boom 16 has a base end attached via a support shaft 17 and can be raised and lowered around the support shaft 17. A hoisting cylinder 18 is interposed between the bracket 15 and the telescopic boom 16, and the telescopic boom 16 is hoisted by expansion and contraction of the hoisting cylinder 18.

  The telescopic boom 16 has a base boom 16A, an intermediate boom 16B, and a tip boom 16C, and is configured to be nested in the base boom 16A in this order from the outside to the inside. The telescopic boom 16 is expanded and contracted by an expansion cylinder (not shown).

  A sheave (not shown) is provided at the distal end portion of the distal end boom 16C, and a wire W is hung on the sheave, and the hook block 19 is suspended by the wire W. A hook 21 is attached to the hook block 19.

  The wire W is wound up or sent out by a winch (not shown).

  A monitoring camera (monocular camera) 100 such as a TV camera is attached to the tip of the tip boom 16C via a damper (not shown), and the monitoring camera 100 has a vertical axis in the tilt direction and the pan direction. It can be tilted at an arbitrary angle. Moreover, the direction of the monitoring camera 100 is prevented from changing due to wind or the like by a damper (not shown).

The inclination (orientation) of the monitoring camera 100 is performed by an operation unit (not shown) provided in the cabin 20. The tilt angle of the monitoring camera 100 is detected by the tilt angle detection sensor S1 and the pan angle detection sensor S2 (see FIG. 2).
[Monitoring camera device]
FIG. 2 is a block diagram illustrating the configuration of the monitoring camera device 110.

The monitoring camera device 110 includes a monitoring camera 100 provided at the distal end of the telescopic boom 16, a boom posture detection sensor 50 that detects the posture of the telescopic boom 16, and a tilt angle detection sensor S1 that detects the tilt angle of the monitoring camera 100. And the pan angle detection sensor S2 and an image processing controller (image arithmetic processing device) for obtaining the height of a structure (representing an object as a structure) imaged by processing an image signal output from the monitoring camera 100 ) 60 and a monitor 51 for displaying an image captured by the monitoring camera 100.
[Boom attitude detection sensor]
The boom posture detection sensor 50 detects the extended length of the telescopic boom 16, the elevation angle of the telescopic boom 16, and the turning angle of the telescopic boom 16. Have.
[Image processing controller]
When the telescopic boom 16 is moved from the storage position to the work start position (standby position) or while the work is moving, the image processing controller 60 is based on two images captured by the monitoring camera 100 at different positions. It is something that asks for height.

As shown in FIG. 3, the image processing controller 60 includes first and second frame memory units 61 and 62 that capture an image captured by the monitoring camera 100, and a building from the first image captured in the first frame memory unit 61. The extraction unit 63 that extracts the feature points of the structure, the image conversion unit 64 that converts the second image captured in the second frame memory unit 62 into a predetermined image, and the conversion converted by the image conversion unit 64 An extraction unit 65 that extracts a feature point P of a structure I such as a building from the image (see FIG. 4) and the optical axes of the first and second images captured by the first and second frame memory units 61 and 62 A calculation unit and 66 for performing various calculations for obtaining a distance, obtaining the height of the structure I from the distance and the data detected by the boom posture detection sensor 50, and the structure obtained by the calculation unit 66 A numerical value indicating the height of I The image synthesizing unit 68 that synthesizes the image of the structure I and the image captured by the monitoring camera 100 are taken into the first and second frame memory units 61 and 62, or the arithmetic unit 66 performs arithmetic processing. A control unit 69 and the like.
[Operation]
Next, the operation of the monitoring camera apparatus 110 configured as described above will be described.

  When the telescopic boom 16 is lifted from the retracted position (not shown) as shown in FIG. 4 and extended to the work position, the surveillance camera 100 captures an image as the telescopic boom 16 extends, and the captured image is displayed. It is displayed on the monitor 51. Here, it is assumed that the monitoring camera 100 is directed directly below.

  When the monitoring camera 100 reaches the chain line position (first position) shown in FIG. 4, the control unit 69 causes the first frame memory unit 61 to capture an image captured by the monitoring camera 100.

  When the image is taken into the first frame memory unit 61, the extracting unit 63 extracts the contour line (edge) of the image of the imaged structure I by image processing, and further, as shown in FIG. A point P1 that is a corner of the structure I in the line is extracted as a feature point by image processing, and the position of the feature point P1 is obtained as a position (X1, Y1) on the image G1.

  When the surveillance camera 100 comes to the solid line position (second position) in FIG. 4 due to the extension of the telescopic boom 16, the control unit 69 causes the second frame memory unit 62 to capture an image captured by the surveillance camera 100.

  When the image is captured in the second frame memory unit 62, the surveillance camera 100 is an image picked up from the position Q1 moved to the right by L (in FIG. 4) at the same height position as the first position. The image conversion unit 64 converts the image in the second frame memory unit 62.

  This image conversion performs geometric conversion based on detection data detected by the boom posture detection sensor 50. If necessary, this converted image is stored in a third frame memory unit (not shown).

The extraction unit 65 extracts image contour lines (edges) of the same building (structure) based on the converted image G2 shown in FIG. 5B by image processing, and further, corners of the contour lines are extracted. A point P2 to be a part is extracted as a feature point by image processing, and the position of the point P2 is obtained as a position (X2, Y2) on the image G2.
[principle]
Here, the principle of obtaining the depth by stereo shooting will be briefly described.

  As shown in FIG. 6, as the coordinate system, the position of the reference camera is set as the origin, and each coordinate axis of XYZ is set. The position of the reference camera is (B, 0, 0), and the distance between both cameras is B.

  The image obtained by the camera is a two-dimensional image in which a target object in three dimensions is projected and projected onto the plane Z = f. f is a focal length.

  The image coordinates (x, y) are two-dimensional coordinates with the image center as the origin O, and the image coordinate axes x and y are set to be parallel to the X axis and Y axis in the three-dimensional space.

Assume that a point D (Xp, Yp, Zp) in the three-dimensional space shown in FIG. 7 is imaged by two cameras. At this time, in the image obtained by the reference camera (left camera), it is reflected at the position U (x _L , y _L ), and in the image by the reference camera (right camera), it is reflected at the point V (x _R , y _R ). It shall be. However, when two identical cameras are arranged in parallel along the X axis, y _L = y _R.

  The depth Z by triangulation is obtained from the similarity of triangles. FIG. 8 is a diagram when the state of FIG. 7 is viewed from above in the Y-axis direction.

  A target point D (Xp, Yp, Zp) in a three-dimensional space and a triangle (ΔDOV) formed by two camera positions are similar to a triangle (ΔOEF) formed by a reference camera position and parallax on the image. Therefore, the following relational expression holds.

| X _L −x _R |: f = B: Zp (1)
x _R −x _L is the parallax on the reference plane, and when this is d, the depth Zp is expressed by the equation (1) as follows:
Zp = −Bf / d (2)
Can be obtained as

In the camera arrangement of FIG. 7, since x _L > x _R is always satisfied and the parallax d is defined by x _R −x _L , a minus sign is added to the above equation.

  If the depth is obtained, the target point D (Xp, Yp, Zp) can be obtained by scaling as follows.

That is, f: x _L = Zp: Xp, and Xp = x _L · Zp / f. Similarly, Yp = y _L · Zp / f.

Therefore,
(Xp, Yp, Zp) = (x _L · Zp / f, y _L · Zp / f, -Bf / d)
= −B / d (x _L , y _L , f) (3)
It becomes. That is, if the position U (x _L , y _L ) of the image captured by the left camera and the position V (x _R , y _R ) of the image captured by the right camera are obtained, the position of the point D from the equation (3) Can be requested. A value in a three-dimensional space is used as the coordinate value of the image coordinates (x, y).

  In this embodiment, as described above, the position (X1, Y1) of the feature point P1 (see FIG. 5) of the image G1 imaged at the first position shown in FIG. 4 and the feature point of the image G2 imaged at the position Q1. The position (X2, Y2) of P2 is obtained. Note that values in the three-dimensional space are used as the coordinate values of the positions (X1, Y1) and (X2, Y2).

  Therefore, if the reference camera is the first position camera and the reference camera is the second position camera and the moving distance L (see FIG. 4) of the monitoring camera 100 in the horizontal direction corresponding to the distance B between the two cameras is obtained, (3) By obtaining the distance h1 from the equation, the height H of the structure I can be obtained.

  The moving distance L of the monitoring camera 100 is obtained from the detection signal detected by the boom posture detection sensor 50 by the calculation unit 66 shown in FIG. That is, the calculation unit 66 obtains the movement distance L based on the elevation angle of the telescopic boom 16 and the extended length.

The calculation unit 66 determines the position of the monitoring camera 100 (see FIG. 4) at the first position from the position U (x _L , y _L ), V (x _R , y _R ) of the image shown in FIG. The vertical distance h1 from the structure to the structure I is obtained based on the above equation (3).

  Further, the calculation unit 66 obtains the height position h0 of the monitoring camera 100 at the first position shown in FIG. 4 from the detection signal detected by the boom posture detection sensor 50, and the structure I is calculated from the distance h1 and the height position h0. Find the height H.

  Here, the height H of the structure I is obtained. However, a feature point of a fixed object such as a material placed on the ground may be extracted, and the height of the fixed object may be calculated by a similar method. .

  As shown in FIG. 9, the combining unit 68, on the image I3 of the structure I displayed on the monitor 51, the black circle mark M indicating the position of the extracted feature points P <b> 1 and P <b> 2 and the obtained structure I. And a numerical value indicating the height H of the image.

  That is, the numerical value indicating the height H is displayed in association with the structure I so that it can be seen that it indicates the height of the structure I. Here, a black circle is displayed at the positions of the feature points P1 and P2, and a numerical value is displayed near the black circle so that the height of the structure I is shown.

  When the monitoring camera 100 is imaging other structures besides the structure I, the feature points of the other structures are extracted in the same manner as described above, and the height of the structure is obtained to determine the height of the monitor 51. Display on the screen.

  As described above, the image captured by the monitoring camera 100 provided at the distal end portion of the telescopic boom 16 is stereo processed by performing image processing on one image having a different imaging position and a different imaging position with the movement of the telescopic boom. Since the height H of the structure I is obtained by obtaining an image similar to that captured by the camera, the height H of the structure I can be obtained by one monitoring camera 100.

  Further, since the surveillance camera 100 is provided at the tip of the telescopic boom 16, the height of the high structure I can be obtained by extending the telescopic boom 16. Further, since the monitoring camera 100 can capture an image from a high position, the height of the structure in the periphery necessary for the work can be obtained, and a three-dimensional diagram in the periphery can be created. For this reason, the safety of work can be improved more.

  By the way, when the contour line and the feature points P1 and P2 are extracted from the captured image, the structure I appears smaller as the height of the monitoring camera 100 becomes higher. By making the object I appear large, the number of pixels for extracting the feature points P1 and P2 is roughened, and the density of the pixels is increased as the height thereof increases, so that the feature points P1 and P2 Extraction can be calculated accurately in a short time. In this case, the positions of the feature points P1 and P2 may be calculated and averaged a plurality of times.

  In the above embodiment, the height of the structure I is obtained from the images taken at the first and second positions. However, the height of the structure I is continuously obtained when the telescopic boom 16 is extended. The height of the structure I may be determined from the obtained distribution states of the plurality of heights.

  If the height is measured when the telescopic boom 16 is turned, the height of the surveillance camera 100 is constant, so that the height of the structure I can be obtained with high accuracy. For this reason, when the telescopic boom 16 is turned after the height of the structure I is obtained, the height of the structure I may be obtained during the turning and the height of the structure I may be updated. .

  When the telescopic boom 16 is lifted and the crane 10 is moved, if the surveillance camera 100 captures an image, the image can be captured from above over a wide range, and the height of the structure within the wide range can be determined. Can be sought. For this reason, it is possible to create a three-dimensional view within the wide range.

  In the above-described embodiment, the case where the monitoring camera 100 that captures images at the first and second positions is directed directly below, but the monitoring camera 100 is tilted or panned at the first position or the second position, The height of the structure I can be obtained by performing image processing so that the surveillance camera 100 is directed directly below and captured.

  In the above embodiment, feature points of the structure are extracted from the captured image and the height of the feature points is obtained. However, the present invention is not limited to this. For example, a touch panel is provided on the screen of the monitor 51 and displayed on the screen. You may make it obtain | require the height of the desired site | part of the image by touching the desired site | part of the image which is on from a touch panel.

  The present invention is not limited to the above-described embodiments, and design changes and additions are permitted without departing from the spirit of the invention according to each claim of the claims.

16 Telescopic boom 51 Monitor 60 Image processing controller (image processing unit)
61 First frame memory unit 62 Second frame memory unit 63 Extraction unit 64 Image conversion unit 65 Extraction unit 66 Calculation unit 68 Composition unit 100 Surveillance camera (monocular camera)

Claims (5)

A surveillance camera device having a monocular camera mounted near the tip of a crane boom,
While the boom is moved from the storage position, an image captured by the monocular camera is captured, and the two first and second images captured and the distance between the optical axes of the first and second images are captured. A surveillance camera device comprising an image arithmetic processing device for obtaining a height of an object being imaged.   The image arithmetic processing device extracts a feature point of an object from the first image, and obtains a second image so as to be an image obtained when the second imaging position is set to a height position of the first imaging position. Converting, extracting the feature points of the object from the converted converted image, and based on the positions of the feature points of the first image and the converted image on the images and the distance between the optical axes. The surveillance camera device according to claim 1, wherein the height of the object is obtained.   The surveillance camera device according to claim 1, wherein the height of the object is continuously obtained during the movement of the boom.   The surveillance camera device according to claim 3, wherein when the boom is turned, the height of the object already obtained is updated to the height obtained during the turn. A monitor that displays an image captured by the monocular camera;
The numerical value indicating the height of the object displayed on the monitor is displayed on the monitor in association with the image of the object so that it can be seen that the height of the object is indicated. Item 5. The surveillance camera device according to any one of items 4.