JP2021075359A - Periphery monitoring device for work vehicle having boom - Google Patents

Abstract

To provide a periphery monitoring device for a work vehicle having a boom that detects a person based on a plurality of images photographed by a plurality of cameras provided on a traveling body while suppressing costs.SOLUTION: A periphery monitoring device 10 of a crane 1 having a boom 7 includes four cameras 11a to 11d provided to photograph the periphery of the crane and a person detection unit 17 for detecting a person H based on images Pa to Pd photographed by the cameras 11a to 11d. The person detection unit 17 includes a revolving position detection unit 17a for detecting a revolving position of the boom 7, an image selection unit 17b for selecting one of the images Pa to Pd having a predetermined positional relationship relative to the revolving position of the boom 7 in accordance with the revolving position of the boom 7, and a parameter adjustment unit 17c for changing a parameter of processing such that detection accuracy of the person H of the processing with respect to the image selected by the image selection unit 17b is increased more than the detection accuracy of the processing with respect to another images that are not selected by the image selection unit 17b.SELECTED DRAWING: Figure 4

Description

The present invention relates to a peripheral monitoring device for a work vehicle having a boom.

Some work vehicles such as cranes and aerial work platforms equipped with a boom are provided with a plurality of cameras to monitor the surroundings of the work vehicle. For example, cameras are provided at the front, right, rear, and left sides of the traveling body of the work vehicle, and these cameras are used to photograph the front, right, rear, and left sides of the traveling body. Images are combined to generate an all-around image corresponding to the entire circumference of the work vehicle, and this all-around image is displayed on a monitor (see, for example, Patent Document 1).

The operator can see this all-around image and confirm that there are no workers or other people around the work vehicle.

JP-A-2018-042205

In addition, the image taken by the camera provided in the vehicle is not simply displayed on the monitor, but the pedestrian is automatically detected based on the taken image, and the detected pedestrian is emphasized. This may alert the driver.

Here, since the above-mentioned person detection for automatically detecting a pedestrian performs processing for each image of each camera, it is common to provide a separate processing unit (CPU) for each camera.

However, it is necessary to bear a high cost if the number of processing units is the same as the number of cameras. On the other hand, it is also conceivable to process a plurality of images taken by a plurality of cameras with a single processing unit. However, in this case, since the load applied to the processing unit becomes high, it is necessary to provide a processing unit having high specifications in order to obtain a desired processing speed, which is costly.

The present invention has been made in view of the above circumstances, and is a work having a boom capable of performing human detection based on a plurality of images taken by a plurality of cameras provided on a traveling body while suppressing costs. An object of the present invention is to provide a vehicle surrounding monitoring device.

The present invention performs a process of detecting a person based on a plurality of cameras provided for photographing the surroundings of a work vehicle having a boom and a plurality of images obtained by photographing with the plurality of cameras. The person detection unit includes a person detection unit, and the person detection unit includes a plurality of the images according to the rotation position detection unit that detects the rotation position of the boom and the rotation position of the boom detected by the rotation position detection unit. Among the images, the image selection unit that selects an image having a predetermined positional relationship with the turning position of the boom, and the processing for the image selected by the image selection unit are performed on other images that are not selected by the image selection unit. It is a peripheral monitoring device for a work vehicle having a boom provided with a parameter adjusting unit that changes processing parameters so as to improve the detection accuracy of the person rather than the processing.

According to the peripheral monitoring device for a work vehicle having a boom according to the present invention, it is possible to suppress the cost and perform human detection based on a plurality of images taken by a plurality of cameras provided on the traveling body.

It is a side view which saw the crane 1 as an example of a work vehicle from the right side. It is a top view of the crane 1 of FIG. It is a schematic diagram which shows an example of the inside of a cabin. It is a block diagram which shows the surroundings monitoring device provided in a crane. It is a schematic diagram which shows the all-around bird's-eye view image. It is a flowchart which shows the flow (action) of the operation of the surroundings monitoring device.

Hereinafter, a specific embodiment of the surrounding monitoring device for a work vehicle having a boom according to the present invention will be described with reference to the drawings.

<Outline configuration of crane>
FIG. 1 is a side view of a crane 1 as an example of a work vehicle viewed from the right side, and FIG. 2 is a plan view of the crane 1 of FIG. 1 viewed from above.

The illustrated crane 1 includes a traveling body (carrier) 2 and a swivel body 3. The traveling body 2 is a portion having a traveling function, and has a plurality of wheels and a driving source such as an engine for driving the wheels and the turning body 3.

Although not shown, the traveling body 2 is provided with a pair of left and right outriggers on the front side and the rear side, respectively, and by projecting the outriggers in the vehicle width direction W and touching the ground, when working with the boom 7. The stability of the traveling body 2 is improved in (work mode).

As shown in FIG. 1, the swivel body 3 is arranged above the traveling body 2 and is provided so as to be able to swivel around the vertical axis C with respect to the traveling body 2. The swivel body 3 has a cabin 5 and a boom support 6. The cabin 5 is a passenger compartment for the operator Op who performs the crane work to perform various operations, and is arranged unevenly on the right side in the vehicle width direction W.

FIG. 3 is a schematic view showing an example of the inside of the cabin 5. Inside the cabin 5, an operation unit corresponding to various operations is provided. Various operations include, for example, turning the swivel body 3, undulating and expanding and contracting the boom 7, hoisting and lowering the wire rope 8 for suspension by a winch drum provided on the boom support 6, and extending and storing the outrigger. There are starting and stopping of the engine, which is the drive source.

Further, inside the cabin 5, a display 14, a controller 13, a buzzer 16 and the like of a surrounding monitoring device 10 (see FIG. 4 described later) described later are arranged.

The boom support 6 is a member that supports the boom 7 extending in the front-rear direction L in a prone state, and the base end portion of the boom 7 is attached to the boom support 6 via a boom root fulcrum pin. The boom 7 is rotatably supported so as to undulate in a vertical plane about a boom root fulcrum pin.

An undulating cylinder is provided between the boom support 6 and the boom 7, and by extending the undulating cylinder, the boom 7 rotates around the boom root fulcrum pin and rises from the prone state, and then rises again. By contracting the undulating cylinder from the state, the undulating operation is performed in the undulating state.

The boom 7 has, for example, a base end boom 7a connected internally by a telescopic cylinder, four-stage intermediate booms 7b, 7c, 7d, 7e, and a tip boom 7f. In the contracted state of the telescopic cylinder, the intermediate booms 7b to 7e are arranged inside the base end boom 7a, and the tip boom 7f is arranged inside the intermediate boom 7e in a nested manner, and the boom 7 is in the contracted state. Will be.

When the telescopic cylinder is extended from the contracted state shown in FIG. 1, each intermediate boom 7b to 7e protrudes from the inside of the base end boom 7a, the tip boom 7f protrudes from the inside of the intermediate boom 7e, and the boom 7 is extended. Become.

A wire rope 8 wound around a winch drum is hung on a sheave arranged at the tip block 7g at the tip of the tip boom 7f, and the wire rope 8 is provided with a hook block 9. Luggage is hung on the hook 9a of the hook block 9 by a slinging tool.

The load hung on the hook 9a can be lifted, moved, and hung by one or more of the undulations and expansions and contractions of the boom 7, the swivel of the swivel body 3, and the hoisting and lowering of the wire rope 8. Served.

<Around monitoring device>
FIG. 4 is a block diagram showing a surrounding monitoring device 10 provided on the crane 1. The surroundings monitoring device 10 provided on the crane 1 photographs the surroundings of the crane 1 with a camera 11, displays the image of the surroundings obtained by the images on the display 14, and makes it visible to the operator Op. , The person present in the image is automatically detected, highlighted and displayed on the display 14, or the buzzer 16 is sounded to alert the operator Op.

As shown in FIG. 4, the ambient monitoring device 10 includes a plurality of cameras 11, a plurality of sensors 12 (moving object detection units), a controller 13, a display 14, and a buzzer 16.

The plurality of cameras 11 are a front camera 11a, a rear camera 11b, a right side camera 11c, and a left side camera 11d. Each of the cameras 11a, 11b, 11c, and 11d is a camera having a fisheye lens having a wide angle of view of about 180 [degrees].

FIG. 5 is a schematic view showing an all-around bird's-eye view image P. As shown in FIG. 2, the front camera 11a is provided at the front portion of the traveling body 2, and mainly captures a front image (front image) Pa of the periphery of the crane 1. The rear camera 11b is provided at the rear of the traveling body 2, and mainly captures a rear image (rear image) Pb around the crane 1.

The right side camera 11c is provided on the right side of the traveling body 2, and mainly captures the right side image (right side image) Pc around the crane 1. The left side camera 11d is provided on the left side of the traveling body 2, and mainly captures the left side image (left side image) Pd around the crane 1.

The photographing area by the front camera 11a and the photographing area by the right side camera 11c overlap in a part corresponding to the front side of the photographing area of the right side camera 11c. The photographing area by the front camera 11a and the photographing area by the left side camera 11d also overlap in a part corresponding to the front side of the photographing area of the left side camera 11d.

The photographing area by the rear camera 11b and the photographing area by the right side camera 11c overlap in a part corresponding to the rear side of the photographing area of the right side camera 11c. The photographing area by the rear camera 11b and the photographing area by the left side camera 11d also overlap in a part corresponding to the rear side of the photographing area of the left side camera 11d.

In this way, the four cameras 11a to 11d photograph the circumference of the crane 1 over the entire circumference.

The plurality of sensors 12 are a front sensor 12a, a rear sensor 12b, a right side sensor 12c, and a left side sensor 12d. Each sensor 12a, 12b, 12c, 12d is a sensor that detects a moving object in a predetermined range. The sensors 12a, 12b, 12c, and 12d are provided corresponding to the cameras 11a, 11b, 11c, and 11d.

That is, the front sensor 12a is provided in the front part of the traveling body 2, and the front side taken by the front camera 11a is set as the detection range of the moving body.

Similarly, the rear sensor 12b is provided at the rear of the traveling body 2 and the rear portion photographed by the rear camera 11b is set as the detection range of the moving object, and the right side sensor 12c is provided at the right side of the traveling body 2 and is on the right side. The right side taken by the camera 11c is the detection range, the left side sensor 12d is provided on the left side of the traveling body 2, and the left side taken by the left side camera 11d is the detection range.

Images taken by the cameras 11a to 11d are input to the controller 13, and the detection results of moving objects detected by the sensors 12a to 12d are input to the controller 13.

The controller 13 converts the four images Pa, Pb, Pc, and Pd taken by the cameras 11a to 11d into bird's-eye view images PA, PB, PC, and PD, which are assumed to look down vertically from above the crane 1. To process. Further, the controller 13 combines the four bird's-eye view images PA to PD obtained by the viewpoint conversion process into a single image (entire-circumferential bird's-eye view image) in which the entire circumference around the crane 1 is connected as shown in FIG. Synthesize processing is performed in P).

Further, the controller 13 includes a person detection unit 17. The person detection unit 17 detects the person H in those images based on the images Pa to Pd or based on the bird's-eye view images PA to PD. The detection of the person H from the image can be performed, for example, by detecting an edge in the image and comparing the contour which is the connection of the edges with the shape of the person H. The human H may be detected by a method other than the above-mentioned contour shape matching or the like.

When the person detection unit 17 detects the person H, the controller 13 generates a rectangular frame F that surrounds the outside of the detected person H, as shown in FIG. Further, when the person detection unit 17 detects the person H, the controller 13 sounds the buzzer 16 provided in the cabin 5.

In this way, the surrounding monitoring device 10 displays the person H in the image surrounded by the rectangular frame F, so that the whole-around bird's-eye view is compared with the case where the person H is simply displayed in the all-around bird's-eye view image P. The person H in the image P can be emphasized, and the presence of the person H can be easily noticed by the operator Op who sees the display 14.

Further, by sounding the buzzer 16 in the surrounding monitoring device 10, the operator Op can confirm the existence of the person H by hearing. Therefore, the surrounding monitoring device 10 can make the presence of the person H notice even if the operator Op does not notice the display of the rectangular frame F surrounding the detected person H when the operator Op is not looking at the display 14.

Here, one person detection unit 17 performs detection processing of the person H on the corresponding four images Pa to Pd or the four bird's-eye view images PA to PD of the four cameras 11a to 11d. Therefore, the processing load of the person detection unit 17 of the present embodiment is larger than that of the person detection unit 17 having a person detection unit corresponding to each of the four cameras 11a to 11d.

Therefore, if the single person detection unit 17 tries to perform accurate detection processing on all four images Pa to Pd or all four bird's-eye view images PA to PD, the processing may not be in time. obtain.

Therefore, the person detection unit 17 of the present embodiment performs equally accurate processing on the four images Pa to Pd or the four bird's-eye view images PA to PD in order to reduce the load of the detection processing of the person H. However, accurate detection processing is performed only on one or two images Pa to Pd or bird's-eye view images PA to PD that are set preferentially, and the remaining two or three images Pa to Pd are performed. Alternatively, the bird's-eye view images PA to PD are subjected to detection processing with reduced accuracy.

Specifically, the person detection unit 17 selects a predetermined one image Pa to Pd (or one bird's-eye view image PA to PD) corresponding to the turning position of the boom 7, and the selected one image Pa to Accurate processing is performed on Pd (or one bird's-eye view image PA to PD), and the selected 1 is applied to the remaining three other images Pa to Pd (or three bird's-eye view images PA to PD). The processing is less accurate than one images Pa to Pd (or one bird's-eye view images PA to PD).

In order to perform such processing, the person detection unit 17 includes a turning position detection unit 17a, an image selection unit 17b, and a parameter adjustment unit 17c.

The swivel position detection unit 17a detects the swivel position (swivel angle) of the boom 7 (swivel body 3). The turning position of the boom 7 is the direction in which the tip of the boom 7 is facing. For example, the turning position when the tip of the boom 7 is facing the front of the traveling body 2 of the crane 1 is 0 [degree], and the traveling body 7 is running. The turning position when facing the right side of the body 2 is 90 [degrees], the turning position when facing the rear of the traveling body 2 is 180 [degrees], and when facing the left side of the traveling body 2. The turning position of is 270 [degrees].

The swivel position detection unit 17a detects the swivel angle of the swivel body 3 of the crane 1 (the angle around the vertical axis C in FIG. 1) based on the input crane information, but is a device for detecting the overload of the crane 1. It may be detected by an existing detector provided in advance for (AML), or may be detected by a newly provided one.

The image selection unit 17b selects an image corresponding to a predetermined positional relationship with respect to the turning position of the boom 7 as an image of interest. The predetermined positional relationship with respect to the turning position of the boom 7 selected by the image selection unit 17b is, for example, a positional relationship of 180 [degrees] with respect to the turning position of the boom 7.

That is, the image selection unit 17b selects one image Pa to Pd (or one bird's-eye view image PA to PD) existing in the direction opposite to the direction in which the tip of the boom 7 faces. The image selection unit 17b of the person detection unit 17 of the present embodiment is behind the swivel body 3 (at the swivel position of the boom 7), which the operator Op operating in the cabin 5 cannot directly visually recognize from the cabin 5. On the other hand, images Pa to Pd (or one bird's-eye view images PA to PD) of 180 [degrees] of the positional relationship are selected.

Then, the parameter adjusting unit 17c refers to one image Pa to Pd (or one bird's-eye view image PA to PD) having a positional relationship of 180 [degrees] with respect to the turning position of the boom 7 selected by the image selection unit 17b. Therefore, the parameters of the process for detecting the person H more accurately than the process for the other three images Pa to Pd (or the other three bird's-eye view images PA to PD) that are not selected are set.

Further, the parameter adjusting unit 17c sets parameters for processing for detecting the person H with respect to the other three images Pa to Pd (or the other three bird's-eye view images PA to PD) not selected by the image selection unit 17b. , The parameters are set so that the accuracy of detecting the person H is lower than the processing for one selected images Pa to Pd (or one bird's-eye view images PA to PD).

The processing parameters set by the parameter adjusting unit 17c are, for example, the period (frame rate [fps]) and the resolution for processing the image captured by the camera 11. That is, by increasing the frame rate as a processing parameter for accurately detecting the person H (for example), 30 [fps], the cycle in which the person H can be detected can be shortened, thereby shortening the period in which the person H can be detected. The accuracy of detecting the person H can be improved.

Further, by increasing the resolution as a processing parameter for accurately detecting the person H, the accuracy of the image capable of detecting the person H can be improved.

In this way, the person detection unit 17 of the present embodiment has one image Pa to Pd (1 degree] in the positional relationship of 180 [degrees] with respect to the turning position of the boom 7 in the all-around bird's-eye view image P of the crane 1. Alternatively, the person H can be detected accurately by using one bird's-eye view image (PA to PD) as the image of interest.

On the other hand, the person detection unit 17 of the present embodiment reduces the accuracy (frame rate) of the three images Pa to Pd (or the three bird's-eye view images PA to PD) other than the image of interest because the degree of attention is low. (For example, 10 [fps]) or the image resolution can be lowered, so that the processing load of the controller 13 can be reduced.

The person detection unit 17 of the present embodiment assumes that the image of interest that enhances the detection accuracy of the person H has a positional relationship of 180 [degrees] with respect to the turning position of the boom 7, but the turning of the boom 7 It may be in a positional relationship of 0 [degrees] with respect to the position, or may be arbitrarily selected by selection by the operator Op.

In this case, for example, the display 14 is used as a touch panel as the reception unit 15, and the operator Op sets one of the four bird's-eye view images PA to PD constituting the all-around bird's-eye view image P displayed on the display 14 as the operator Op. Can be selected as a attention image of one selected bird's-eye view images PA to PD by touching and selecting the reception unit 15 as a attention image for increasing the detection accuracy of the person H.

In this case, the two bird's-eye view images PA to PD can be selected as the attention image.

Further, as described above, any one of the images Pa to Pd (or one bird's-eye view image PA to PD) is selected as the image of interest automatically or arbitrarily by the operator Op according to the turning position of the boom 7. Even if it is, when any of the sensors 12a to 12d detects a moving object, the image selection unit 17b detects the moving object from the existing selected image by the image selection unit 17b. One image corresponding to the detection range of 12a to 12d (image taken by the cameras 11a to 11d corresponding to the installation position of the sensors 12a to 12d) may be switched to Pa to Pd.

If a moving object is detected in the image around the crane 1 where the operator Op has a low degree of attention, the moving object may be a human H. Therefore, the image is displayed as a human H by a process with good detection accuracy. By detecting, the operator Op can quickly switch the image of interest.

FIG. 6 is a flowchart showing an operation flow (action) of the surrounding monitoring device 10. The operation of the ambient monitoring device 10 of the present embodiment described above will be described with reference to the flowchart of FIG.

The images Pa to Pd taken from the cameras 11a to 11d are input to the person detection unit 17 of the controller 13 (S1). Further, the turning position of the boom 7 is input to the person detection unit 17 as crane information (S2).

The swivel position detection unit 17a of the person detection unit 17 detects the swivel position of the input boom 7, and the image selection unit 17b refers to the detection position of the boom 7 among the four input images Pa to Pd. One image Pa to Pd having a preset positional relationship is selected (S3).

When the operator Op selects the images Pa to Pd through the reception unit 15 (YES in S4), the image selection unit 17b selects the images Pa to Pd selected in S3 by the operator Op. Switch to ~ Pd (S5).

When the operator Op does not select the images Pa to Pd through the reception unit 15 (NO in S4) and when the selected image is switched (S5), the moving object is detected by any of the sensors 12a to 12d. It is determined whether or not there is an input indicating that (S6).

When there is an input indicating that a moving object has been detected (YES in S6), the image selection unit 17b uses the images Pa to Pd selected in S3 or the images Pa to Pd switched in S5 as a sensor for detecting the moving object. The images Pa to Pd of the cameras 11a to 11b corresponding to 12a to 12d are switched (S7).

When there is no input indicating that a moving object has been detected (NO in S6), the image selected in S3, S5, or S7 is set as the image of interest.

The person detection unit 17 performs a process of detecting the person H based on all the images Pa to Pd (or the bird's-eye view images PA to PD) (S9). At this time, the parameter for the process of detecting the person H is performed. The parameter adjusting unit 17c for adjusting the above adjusts the selected images Pa to Pd (or one bird's-eye view image PA to PD) to the other three images Pa to Pd (or three bird's-eye view images PA) that are not selected. -PD), the parameters are adjusted so that the detection process of the person H is a more accurate process (S8).

On the contrary, the parameter adjusting unit 17c has the selected one image Pa to Pd (or one bird's-eye view image PA to PD) for the three unselected images Pa to Pd (or three bird's-eye view images PA to PD). ), The parameter is adjusted so that the detection process of the person H is a process with lower accuracy (S8), and the person detection unit 17 performs the detection process of the person H (S9).

When the person H is detected by the person detection unit 17 with the corresponding processing parameters based on the images Pa to Pd (also the bird's-eye view images PA to PD) (YES in S10). In the single all-around bird's-eye view image P obtained by synthesizing the four bird's-eye view images PA to PD, the person detection unit 17 superimposes a rectangular frame F surrounding the detected person H on the display 14 An all-around bird's-eye view image P on which the rectangular frame F is superimposed is output (S11).

The person detection unit 17 sounds the buzzer 16 (S12) in parallel with outputting the all-around bird's-eye view image P on which the rectangular frame F is superimposed to the display 14, and ends when the process ends (YES in S13). If it does not end (NO in S13), the process returns to S1 and the process is repeated.

When the person detection unit 17 does not detect the person H (NO in S10), it ends when the process ends (YES in S13), and when it does not end (NO in S13), it returns to S1 and processes. repeat.

The surrounding monitoring device 10 of the present embodiment adjusts the processing parameters of the attention image so that the detection accuracy of the person H is better than that of the other images, but includes other images excluding the attention image. When a situation occurs in which the processing for detecting the person H cannot be completed in time for the entire image, the frame rate and the resolution of the image may be adjusted according to the turning speed of the boom 7 so that the processing can be performed in time. ..

In the ambient monitoring device 10 of the present embodiment, the image displayed on the display 14 is an all-around bird's-eye view image P, but in the ambient monitoring device according to the present invention, the image displayed on the display is limited to the all-around bird's-eye view image. Instead, the normal images Pa to Pd taken by the camera 11 before the viewpoint conversion process may be used. In this case, the person detection unit 17 may perform a process of detecting the person H on these normal images Pa to Pd.

The surrounding monitoring device 10 of the present embodiment sounds the buzzer 16 when the person H is detected by the person detection unit 17, but the buzzer 16 may be omitted.

1 Crane 7 Boom 10 Surrounding monitoring device 11, 11a, 11b, 11c, 11d Camera 12, 12a, 1b, 12c, 12d Sensor 13 Controller 14 Display 15 Reception 16 Buzzer 17 Person detection 17a Swivel position detection 17b Image selection 17c Parameter adjustment unit F Rectangular frame H Person L Front-back direction Op Operator P All-around bird's-eye view image PA, PB, PC, PD Bird's-eye view image Pa, Pb, Pc, Pd Image W Vehicle width direction L Front-back direction

Claims (3)

Multiple cameras provided to photograph the surroundings of a work vehicle with a boom,
A person detection unit that performs a process of detecting a person based on a plurality of images obtained by taking pictures with the plurality of cameras, respectively, is provided.
The person detection unit
A swivel position detection unit that detects the swivel position of the boom,
An image selection unit that selects an image having a predetermined positional relationship with the rotation position of the boom among a plurality of the images according to the rotation position of the boom detected by the rotation position detection unit.
A parameter adjustment unit that changes the processing parameters so that the processing for the image selected by the image selection unit is higher than the processing for other images not selected by the image selection unit so as to improve the detection accuracy of the person. And, a work vehicle surroundings monitoring device with a boom. It is equipped with a reception unit that accepts inputs that specify the predetermined positional relationship.
The image selection unit is a peripheral monitoring device for a work vehicle having a boom according to claim 1, wherein an image having a predetermined positional relationship is selected based on the designation input to the reception unit. It is provided with a plurality of moving object detecting units provided corresponding to the installation positions of the plurality of cameras, which detect moving objects in a range to be photographed by the plurality of cameras.
When any one of the plurality of moving object detection units detects a moving object, the image selection unit is placed at an installation position where the moving object detecting unit that has detected the moving object is provided among the plurality of cameras. The peripheral monitoring device for a work vehicle having a boom according to claim 1 or 2, wherein an image obtained by taking a picture with a corresponding camera is selected.

Images