JP2020193503A - Operation support system of work machine, operation support method of work machine, maintenance support method of operation support system, and construction machine - Google Patents

Abstract

To provide an operation support system of a work machine capable of reducing burden of an operator in a problem of operating without contact with an obstacle.SOLUTION: An operation support system 1 comprises an acquisition portion 10 and an operation control portion 20. The acquisition portion 10 acquires image information related to a movement range of a movement portion 40 of a work machine 100. The operation control portion 20 makes the movement portion 40 operate so as to avoid contact with an obstacle based on the image information acquired by the image information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a work machine maneuvering support system, a work machine maneuvering support method, a maneuvering support system maintenance support method, and a construction machine.

Construction machines equipped with sensors that detect surrounding workers as obstacles are known. For example, Patent Document 1 describes a construction machine including a lower traveling body, an upper turning body, a monitoring area setting means, an obstacle detection sensor, a calculation means, a time information acquisition means, and a storage means. This construction machine sets a monitoring area based on the relative angle between the lower traveling body and the upper turning body, calculates the position coordinates of obstacles in that area, and stores the calculation result in association with the time information. ..

JP-A-2017-201114

The present inventors have obtained the following recognition about a construction machine provided with a boom arm and an attachment driven by power such as flood control.

A construction machine uses power to drive an arm mechanism such as a boom or an arm, and moves an attachment such as a bucket attached to the arm mechanism to perform a predetermined work. When moving the attachment, the operator of this machine must carefully operate the attachment and arm mechanism so that they do not touch obstacles such as workers. This is a heavy burden on the operator and is a factor in reducing work efficiency.

In order to supplement the visibility, it is conceivable to display the image taken by the camera attached to the construction machine on the display in the cockpit. However, in this case, the maneuvering is performed while checking the display image and the work content at the same time, which increases the burden on the operator and cannot be said to improve the maneuverability.

From the viewpoint of reducing the burden on the operator, it cannot be said that the construction machinery described in Patent Document 1 is sufficiently dealt with. Such challenges can arise not only for construction machinery but also for other types of work machinery.

The present invention has been made in view of these problems, and one object of the present invention is to provide a maneuvering support system for a work machine capable of reducing the burden on the operator.

In order to solve the above problems, the operation support system for a work machine according to an aspect of the present invention includes an acquisition unit that acquires image information regarding the movement range of the moving part of the work machine, and an obstacle in the moving part based on the image information. It is provided with an operation control unit that performs an operation of avoiding contact with.

It should be noted that any combination of the above, and those in which the components and expressions of the present invention are mutually replaced between methods, devices, programs, temporary or non-temporary storage media on which programs are recorded, systems, and the like are also used. It is effective as an aspect of the present invention.

According to the present invention, it is possible to provide a control support system for a work machine that can reduce the burden on the operator.

It is a side view which shows typically the operation support system of the work machine which concerns on 1st Embodiment. It is a block diagram which shows typically the maneuvering support system of FIG. It is a top view which shows the maneuvering support system of FIG. It is a figure which shows typically the front view of the operator of the control support system of FIG. It is a figure for demonstrating the person identification part of the maneuvering support system of FIG. It is a figure for demonstrating the moving part identification part of the control support system of FIG. It is a figure for demonstrating the distance specifying part of the maneuvering support system of FIG. It is a figure for demonstrating the avoidance control part of the control support system of FIG. It is a figure for demonstrating the contact estimation part of the control support system of FIG. It is a flowchart which shows the avoidance process of the control support system of FIG.

Hereinafter, the present invention will be described based on a preferred embodiment with reference to each drawing. In the embodiments and modifications, the same or equivalent components and members are designated by the same reference numerals, and redundant description will be omitted as appropriate. In addition, the dimensions of the members in each drawing are shown enlarged or reduced as appropriate for easy understanding. In addition, some of the members that are not important for explaining the embodiment in each drawing are omitted and displayed.

Also, terms including ordinal numbers such as 1st and 2nd are used to describe various components, but this term is used only for the purpose of distinguishing one component from other components. The components are not limited by.

[First Embodiment]
The configuration of the operation support system 1 of the work machine according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a side view schematically showing the operation support system 1 of the work machine according to the first embodiment. FIG. 2 is a block diagram schematically showing the maneuvering support system 1. FIG. 3 is a plan view schematically showing the maneuvering support system 1.

As shown in FIGS. 1 and 2, the maneuvering support system 1 includes a work machine 100, an acquisition unit 10, and an operation control unit 20. The acquisition unit 10 and the motion control unit 20 form a steering support device 30. The work machine 100 of the present embodiment is a construction machine 1000 that performs construction work by moving the bucket 46. The work machine 100 has a lower traveling portion 36, an upper vehicle body portion 34, an arm mechanism 48, and a bucket 46. The lower traveling portion 36 is configured to be able to travel in a predetermined direction by an endless track or the like. The upper vehicle body portion 34 is mounted on the lower traveling portion 36. The upper vehicle body portion 34 is configured to be rotatable around a vertical axis with respect to the lower traveling portion 36 by a turning drive unit 60. The swivel drive unit 60 can be composed of, for example, a swivel motor (not shown) and a swivel gear (not shown). A cockpit 38 is provided in the upper body portion 34.

For convenience of explanation, the right side of the cockpit 38 is referred to as "right" and the left side is referred to as "left" when viewed from the rear. Further, the front side of the cockpit 38 is referred to as "front", and the opposite direction is referred to as "rear".

The base end portion of the arm mechanism 48 is provided on the right side of the cockpit 38 in the upper vehicle body portion 34. The arm mechanism 48 includes a boom 42 and an arm 44 extending forward from the upper body portion 34. A bucket 46 is attached to the tip end side of the arm mechanism 48. The boom 42 is configured so that the tip portion can rotate up and down around the base end portion on the upper vehicle body portion 34 side. The tip of the arm 44 is configured to be rotatable back and forth around the base end on the boom 42 side.

The bucket 46 is configured such that the tip end portion can be rotated back and forth or up and down around the base end portion on the arm 44 side. The boom 42, arm 44, and bucket 46 can be deformed by changing the bending angle of the joint portion by a plurality of hydraulic cylinders 56. The bucket 46 can be moved by deforming these. Hereinafter, the boom 42, the arm 44, and the bucket 46 are collectively referred to as a moving portion 40. The boom 42, arm 44, and bucket 46 may be referred to as a front portion.

A cockpit 32 is provided inside the cockpit 38. On the side surface of the cockpit 38, a front window 38f surrounding the cockpit 32, a right window 38g, a left window 38h, and a rear window 38j are provided. In the vicinity of the cockpit 32 of the cockpit 38, an operation input unit 54 for manipulating the moving unit 40 by a plurality of levers or the like is provided. In the driver's seat 32, the operator operates the operation input unit 54 while visually observing the surroundings from the windows 38f to 38j to operate the moving unit 40.

When an operation is input from the operation input unit 54, the plurality of hydraulic valves 58 open and close according to the operation. The hydraulic oil supplied from the hydraulic pump (not shown) is sent to the plurality of hydraulic cylinders 56 according to the opening and closing of the hydraulic valve 58. The plurality of hydraulic cylinders 56 expand and contract according to the amount of hydraulic oil delivered, and deform the moving portion 40 to move the boom 42, the arm 44, and the bucket 46. Further, when an operation related to turning is input from the operation input unit 54, the turning drive unit 60 integrally turns the upper vehicle body unit 34 and the moving unit 40 in response to the operation. In this way, the boom 42, the arm 44, and the bucket 46 can move three-dimensionally in a moving range by deforming and turning according to the control of the operator.

In the present embodiment, as shown in FIG. 3, the movable three-dimensional range of the moving unit 40 when the operation input unit 54 is operated is defined as the moving range Rm.

FIG. 4 is a diagram schematically showing the front view of the operator. As shown in this figure, the right side of the front view is blocked by the arm mechanism 48. The cockpit 38 and the cockpit 32 are arranged closer to the left side of the upper vehicle body portion 34 so as to secure the field of view of the tip portion of the bucket 46. In this case, the field of view around the tip of the bucket 46 can be secured, but the field of view on the right side of the bucket 46 is insufficient.

In this way, the driver's forward view is blocked by the arm mechanism 48. That is, as shown in FIG. 3, the moving unit 40 forms a large blind spot region Az in the moving range Rm seen from the driver's seat 32. Therefore, when moving the bucket 46 within the blind spot region Az, the operator needs to steer while changing his / her posture, such as leaning out from the driver's seat, which imposes a heavy burden on the operator. It is also possible for the pilot to overlook obstacles.

In order to supplement the front view, it is conceivable to display an image of the blind spot region Az on a display arranged near the driver's seat 32. However, in this case, the pilot is operated while checking the image on the display and the work content at the same time, and the burden on the operator is rather increased.

Therefore, the maneuvering support system 1 of the present embodiment includes an acquisition unit 10 and an operation control unit 20 in order to reduce the burden on the operator. The acquisition unit 10 acquires the first image information Gp regarding the movement range Rm of the moving unit 40. The motion control unit 20 causes the moving unit 40 to perform an avoidance operation to avoid contact with an obstacle based on the first image information Gp. Since the moving unit 40 automatically performs the avoidance operation, the burden on the operator can be reduced.

The acquisition unit 10 will be described. The acquisition unit 10 includes an image sensor 12 capable of capturing an image in order to acquire the first image information Gp. The image sensor 12 uses an image sensor or the like to image a part or all of the movement range Rm. It is possible to provide only one image sensor 12, but in this case, it is difficult to eliminate the blind spot caused by the moving portion 40. Therefore, the acquisition unit 10 may include a plurality of image sensors 12 that image the moving unit 40 from different positions and directions. In the example of FIG. 3, two image sensors 12-A and 12-B are provided which are arranged apart from each other on the left and right sides of the moving portion 40. In this figure, the symbols Ad-A and Ad-B indicate the imaging range of the image sensors 12-A and 12-B.

Since the two image sensors 12 are arranged on the left and right sides of the moving portion 40, the imaging range Ad can complement each other and the blind spot can be significantly reduced. That is, the two image sensors 12 can be arranged so that the region that becomes a blind spot for one does not become a blind spot for the other. In the example of FIG. 3, the imaging ranges Ad-A and Ad-B are fan-shaped regions that partially overlap in the vicinity of the moving portion 40 and extend to the right and left from there.

The two image sensors 12-A and 12-B may be arranged symmetrically with the moving portion 40 in between. In this case, it is easy to combine the imaging results of the two sensors and integrate them. The two image sensors 12-A and 12-B may be arranged asymmetrically. The directions of the two image sensors 12-A and 12-B along the center of the imaging range Ad (hereinafter, referred to as “visual field direction”) may be parallel or non-parallel. In the example of FIG. 3, the visual field direction of the image sensor 12-A is tilted to the right with respect to the front-rear direction, and the visual field direction of the image sensor 12-B is tilted to the left with respect to the front-rear direction. In this case, the imageable range can be widened to the left and right as compared with the case where the viewing directions are parallel.

By providing the two image sensors 12-A and 12-B, the difference between the imaging results of the two sensors is calculated, and the position and distance between the moving unit 40 and the person 8 are specified with high accuracy from the difference. can do. In addition, since the measurement field of view is expanded by using two image sensors, when an object suddenly enters the turning range when turning at high speed, or a person intrudes into the back side of the moving portion 40. Even in the case, the avoidance operation can be performed quickly. Moreover, since parallax can be obtained, three-dimensional image information can be obtained.

The arrangement height of the acquisition unit 10 is not limited, but a position where the blind spot can be reduced is desirable. In the example of FIG. 1, the acquisition unit 10 is arranged at a position higher than the roof of the cockpit 38. In this case, it is easy to image the ground on which the bucket 46 works. The arrangement heights of the plurality of image sensors 12 may be the same as or different from each other.

It is possible not to include the image information about the moving unit 40 in the first image information Gp, but in this case, it is difficult to specify the distance between the moving unit 40 and the obstacle. Therefore, in the acquisition unit 10 of the present embodiment, the first image information Gp is configured to include the image information regarding the moving unit 40. Specifically, one of the two image sensors 12 is arranged so as to image an area including the left side surface of the moving unit 40, and the other image an area including the right side surface of the moving unit 40. Thereby, even if an obstacle exists on either the left or right side of the moving unit 40, the distance between the moving unit 40 and the obstacle can be specified from the imaging result of one of the two image sensors 12.

The operation control unit 20 will be described with reference to FIG. Each functional block shown in FIG. 2 can be realized by an electronic element such as a computer CPU or a mechanical component in terms of hardware, and can be realized by a computer program or the like in terms of software. It depicts a functional block realized by cooperation. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

As shown in FIG. 2, the operation control unit 20 includes an image information generation unit 20a, an identification unit 20b, a distance identification unit 20d, a storage unit 20m, an avoidance control unit 20f, a valve control unit 20g, and a contact estimation unit. It has 20p and an information output unit 20e. The image information generation unit 20a integrates the first image information Gp of the plurality of image sensors 12 of the acquisition unit 10 to generate the second image information Gs. The valve control unit 20g can control the opening and closing of the hydraulic valve 58 based on the control of the avoidance control unit 20f.

The identification unit 20b sequentially recognizes an obstacle such as a person, the moving unit 40, and the main body of the work machine. For example, the changed region and the unchanged region can be classified from the difference from the second image information Gs generated in the past, and the work machine main body can be recognized from the unchanged region. Further, the obstacle and the moving portion 40 can be recognized separately according to the difference in the change pattern of the changed region. By sequentially performing such recognition processing, it is possible to learn the change pattern of the obstacle, the moving unit 40, and the main body of the work machine, and improve the recognition accuracy.

The identification unit 20b includes an obstacle identification unit 20c, a person identification unit 20h, a moving unit identification unit 20j, and a feature DB 20k. The feature DB 20k is a database in which feature information Fi regarding obstacles is stored in advance. The obstacle identification unit 20c identifies the type of obstacle based on the second image information Gs generated by the image information generation unit 20a and the feature information Fi stored in advance in the feature DB 20k. Hereinafter, the case where the obstacle is a person will be described as an example, but this description can be similarly applied to obstacles other than humans.

The person identification unit 20h will be described. FIG. 5 is a diagram for explaining the person identification unit 20h. If each part of a person is identified in detail, the calculation time becomes long. It is disadvantageous in terms of cost if a high-speed arithmetic element is provided in order to shorten the arithmetic time. Therefore, in the present embodiment, the characteristic points of a person are extracted, and a predetermined range based on the characteristic points is specified as the range of the person.

In the example of FIG. 5, the foot 8b of the person 8 is a feature point. The foot 8b of the person 8 can be easily identified from the contrast with the ground and the characteristics of the shape. In particular, when the obstacle is a person 8, the person identification unit 20h of the present embodiment identifies the foot 8b of the person 8 and identifies the area from the foot 8b to a predetermined height Hh as the range of the person 8. Further, the person identification unit 20h may set a cylindrical range having a predetermined diameter Dh centered on the foot 8b as the range of the person 8. As an example, the height Hh may be 2 m and the diameter Dh may be 1 m. In this case, a range of people can be specified in a practical calculation time by using an inexpensive calculation element.

The range of the person 8 may be calculated based on the image information including the foot 8b, or may be calculated based on the image information other than the foot 8b of the person 8. For example, the range of the person 8 may be calculated based on skeletal information other than the foot 8b of the person 8 (information excluding the feet of the person, etc.).

The moving unit identification unit 20j will be described. FIG. 6 is a diagram for explaining the moving unit identification unit 20j. The moving unit identification unit 20j identifies the moving unit 40 and the main body of the work machine. In particular, the moving unit identification unit 20j identifies the range of the moving unit 40 by using the coordinate information about the predetermined portion of the moving unit 40 stored in advance.

The coordinate information will be described. Measurement points 14a, 14b, and 14c are preset in the boom 42, arm 44, and bucket 46 of the moving unit 40, respectively. The measurement points 14a to 14c may be set to a portion having an existing specific shape such as a bolt or a hole. In the example of FIG. 6, the measurement points 14a to 14c use the marks provided on the boom 42, the arm 44, and the bucket 46.

The moving unit identification unit 20j specifies the range of the work machine 100 based on the reference coordinate information Zp and the acquired coordinate information Zs. The reference coordinate information Zp is coordinate information specified from the second image information Gs generated in the past. The acquired coordinate information Zs is coordinate information specified from the current second image information Gs acquired at any time. For example, the reference coordinate information Zp can be specified from the second image information Gs generated in a state where the work machine 100 is in the standard posture. The specified reference coordinate information Zp is stored in the storage unit 20m.

Further, in order to specify the range of the work machine 100, the storage unit 20m stores the outer shape information Ei regarding the outer shape coordinates of the work machine 100 generated from the design information of the work machine 100. The range at the time of acquisition (current) of the work machine 100 can be specified from the external shape information Ei and the acquired coordinate information Zs. Further, the moving unit identification unit 20j can specify the posture, position, and range of the moving unit 40 by calculation at any time from the relative positions and angles of the measurement points 14a to 14c based on the acquired coordinate information Zs. For example, this calculation can be performed by using the difference between the acquired coordinate information Zs and the reference coordinate information Zp.

The distance specifying unit 20d will be described. FIG. 7 is a diagram for explaining the distance specifying unit 20d. The distance specifying unit 20d specifies the relative shortest distance between the obstacle and the moving unit 40 by calculation. For example, the distance specifying unit 20d is the shortest relative from the person 8 to the moving unit 40 according to the range of the person 8 specified by the person identifying unit 20h and the range of the moving unit 40 specified by the moving unit identifying unit 20j. The distance Ds can be specified. In the example of FIG. 7, the relative distance Ds from the person 8 to the moving portion 40 is specified by the plane coordinates of the measurement points 14a to 14c and the person 8 and the angle from the image sensor 12.

The avoidance control unit 20f will be described. FIG. 8 is a diagram for explaining the avoidance control unit 20f. When the relative distance Ds is specified, the contact between the person 8 and the moving unit 40 can be avoided by various methods. As an example, in the present embodiment, the avoidance control unit 20f changes the movement mode of the moving unit 40 so as to perform a plurality of avoidance operations according to the relative distance Ds between the person 8 and the moving unit 40.

In the present embodiment, the avoidance control unit 20f controls the moving unit 40 to reduce the moving speed of the moving unit 40 when the moving unit 40 approaches the predetermined first distance D1 from the person 8 (avoidance operation 1). For example, when the relative distance Ds is the first distance D1 or less, braking may be applied so that the moving speed of the moving unit 40 becomes a predetermined speed or less. The avoidance control unit 20f can control the turning drive unit 60 to reduce the turning speed of the upper vehicle body unit 34. Further, the avoidance control unit 20f can reduce the rotation speed of the boom 42, the arm 44 and the bucket 46 of the moving unit 40 by controlling the hydraulic valve 58 via the valve control unit 20g. By decelerating the moving unit 40 to a slow speed, it can be stopped immediately when a dangerous state occurs.

In the hydraulic valve 58, when the control of the valve control unit 20g and the control of the operation input unit 54 conflict with each other, the control of the valve control unit 20g may be prioritized. The same applies to the following controls.

In the present embodiment, the avoidance control unit 20f controls the moving unit 40 to stop moving when the moving unit 40 approaches the predetermined second distance D2 from the person 8 (avoidance operation 2). For example, when the relative distance Ds is equal to or less than the second distance D2, the movement of the moving unit 40 may be stopped. In this case, the avoidance control unit 20f may stop the rotation of the upper vehicle body unit 34 and also stop the rotation of the boom 42, the arm 44 and the bucket 46 of the moving unit 40.

Even if a braking force is applied to turning or rotation, due to the inertia of the upper vehicle body portion 34 and the moving portion 40, these may not stop immediately and contact with the person 8 may not be avoided. Therefore, in the present embodiment, the avoidance control unit 20f controls so that the movement course of the movement unit 40 is changed when the movement unit 40 approaches the person 8 within a predetermined third distance D3 (avoidance operation 3). For example, when the relative distance Ds is the third distance D3 or less, the moving course may be changed by raising the moving portion 40, particularly the bucket 46. In this case, the avoidance control unit 20f raises the boom 42, the arm 44, and the bucket 46 by braking the turning drive unit 60 of the upper vehicle body unit 34 and controlling the hydraulic valve 58 via the valve control unit 20g. You may let me.

The third distance D3 may be smaller than the second distance D2, and the second distance D2 may be smaller than the first distance D1. That is, they may have a relationship of D3 <D2 <D1. These distances can be set by experiments, simulations, and the like.

The contact estimation unit 20p will be described. FIG. 9 is a diagram for explaining the contact estimation unit 20p. The contact estimation unit 20p obtains an estimated movement course by simulating these movement courses by calculation according to the coordinates, movement direction, and movement speed of the movement unit 40 and the person 8. FIG. 9A schematically shows the estimated movement course of the moving unit 40 and the person 8. This figure shows each estimated movement course on the coordinates in the left-right direction and the front-back direction. The contact estimation unit 20p obtains the estimated separation distance Dq of the moving unit 40 and the person 8 for each time based on the estimated moving course of the moving unit 40 and the person 8. FIG. 9B shows the change in the estimated separation distance Dq with respect to time.

The avoidance control unit 20f performs a contact prevention operation to prevent contact between the moving unit 40 and the person 8 according to the estimated separation distance Dq. For example, the avoidance control unit 20f executes a contact prevention operation when the estimated separation distance Dq is predicted to be equal to or less than a predetermined threshold value D4. This contact prevention operation may be an operation of decelerating the moving unit 40, an operation of stopping the movement of the moving unit 40, or an operation of changing the moving course of the moving unit 40. Further, a plurality of threshold values D4 may be set, and the content of the contact prevention operation may be changed based on the estimated separation distance Dq and the plurality of threshold values D4.

The information output unit 20e outputs information regarding the distance between the obstacle and the moving unit 40 internally or externally. As an example, the information output unit 20e may output information such as the presence / absence of an obstacle, the relative distance Ds between the moving unit 40 and the person 8, the estimated separation distance Dq, and information related to the avoidance operation to the information terminal 50. The information terminal 50 may display such information in characters or graphics. The information terminal 50 may be provided in the cockpit 38. Further, the information output unit 20e may output a predetermined alarm as sound or light based on the information.

Next, the process S70 related to avoidance control by the acquisition unit 10 and the operation control unit 20 will be described with reference to the flowchart of FIG. Here, an example in which an obstacle is a person will be described.

When the process S70 is started, the acquisition unit 10 acquires image information regarding the movement range Rm of the movement unit 40 (step S71).

Next, the motion control unit 20 identifies an obstacle from the image information acquired by the acquisition unit 10 (step S72).

When there is no obstacle (N in step S72), the motion control unit 20 returns the process to step S71 and repeats the process from step S71. When there is an obstacle (Y in step S72), the motion control unit 20 advances the process to step S73.

Next, the motion control unit 20 identifies the type of obstacle by the obstacle identification unit 20c (step S73).

Next, the motion control unit 20 identifies the range of a person when the obstacle is a person by the person identification unit 20h (step S74).

Next, the motion control unit 20 identifies the range of the moving unit 40 by the moving unit identifying unit 20j (step S75).

Next, the motion control unit 20 specifies the relative shortest distance Ds between the person and the moving unit 40 by the distance specifying unit 20d (step S76).

Next, the motion control unit 20 causes the moving unit 40 to perform an avoidance operation according to the magnitude of the distance Ds by the avoidance control unit 20f (step S77).

When the distance Ds is larger than the first distance D1 (A in step S77), the motion control unit 20 returns the process to step S71 and repeats the process from step S71.

When the distance Ds is equal to or less than the first distance D1 (B in step S77), the motion control unit 20 reduces the moving speed of the moving unit 40 (step S78).

When the distance Ds is the second distance D2 or less (C in step S77), the motion control unit 20 stops the movement of the moving unit 40 (step S79).

When the distance Ds is the third distance D3 or less (D in step S77), the motion control unit 20 changes the movement course of the movement unit 40. (Step S80).

When steps S78, S79, and S80 are completed, the operation control unit 20 returns the process to step S71 and repeats the process from step S71. The process S70 is merely an example, and other steps may be added, some steps may be changed or deleted, or the order of the steps may be changed.

Next, the features of the operation support system 1 of the work machine according to the first embodiment of the present invention will be described. The maneuvering support system 1 causes the acquisition unit 10 to acquire the image information Gp related to the movement range Rm of the moving unit 40 of the work machine 100 and the moving unit 40 to avoid contact with an obstacle based on the image information Gp. It includes an operation control unit 20.

According to this configuration, since the moving unit 40 automatically avoids the contact with the obstacle, the maneuvering becomes easy and the burden on the operator can be reduced. In addition, since safety is ensured, the moving speed of the moving unit 40 can be increased to improve work efficiency.

The acquisition unit 10 may be configured to include image information about the moving unit 40 in the image information Gp. In this case, since the moving portion 40 and the obstacle can be integrally captured in the image information Gp, it is easy to grasp the separation distance with high accuracy.

The acquisition unit 10 may include a plurality of image sensors 12 that image the moving unit 40 from different positions. In this case, since a plurality of image sensors 12 are used, the blind spot of the image sensor 12 can be reduced. In addition, the position and range of the moving portion 40 and the obstacle can be grasped three-dimensionally, and the obstacle detection omission can be prevented. Further, since the measurement field of view is expanded by the plurality of image sensors 12, even if an object suddenly enters the turning range or a person intrudes into the back side of the moving portion 40 when turning at high speed, the measurement can be performed quickly. Avoidance operation is possible.

The motion control unit 20 may control the moving unit 40 to reduce the moving speed of the moving unit 40 when the moving unit 40 approaches within the first distance D1 from the obstacle. In this case, since the moving speed is reduced, it is easy to avoid contact with an obstacle.

The motion control unit 20 may control the moving unit 40 to stop moving when the moving unit 40 approaches within the second distance D2 from the obstacle. In this case, since the moving portion 40 is stopped from moving, it is easier to avoid contact with an obstacle.

The motion control unit 20 may control the moving unit 40 to change the moving course of the moving unit 40 when the moving unit 40 approaches within the third distance D3 from the obstacle. In this case, even if the vehicle cannot be stopped suddenly due to inertia, the moving course of the moving unit 40 can be changed to avoid contact.

The motion control unit 20 may have an obstacle identification unit 20c that identifies the type of obstacle by using the feature information about the obstacle stored in advance. In this case, the avoidance method and the range can be selected according to the type of the obstacle whose type is estimated.

When the obstacle is a person, the person identification unit 20h may be provided to identify the foot 8b of the person 8 and identify the area from the foot 8b to a predetermined height as the range of the person 8. In this case, since the foot 8b can be easily identified from the contrast with the ground and the characteristics of the shape, an appropriate avoidance method and range can be selected by setting the range from the foot 8b to the height corresponding to the height. The motion control unit 20 may include a person identification unit 20h. In this case, the configuration can be made more compact than in the case where the person identification unit 20h is provided separately from the operation control unit 20.

The moving unit identification unit 20j may be provided to identify the range of the moving unit 40 by using the coordinate information about the predetermined portion stored in advance of the moving unit 40. In this case, by setting the feature point as a preset portion, identification becomes easy, and the range of the moving portion 40 can be estimated at high speed or with high accuracy. The delay of avoidance operation can be reduced. The motion control unit 20 may include a moving unit identification unit 20j. In this case, the configuration can be made more compact than in the case where the moving unit identification unit 20j is provided separately from the operation control unit 20.

The moving unit 40 is configured to move according to the operation of the operator, and the moving unit 40 may form a blind spot region in the moving range Rm as seen from the driver's seat 32 of the work machine 100. In this case, since the work machine 100 autonomously avoids contact with obstacles, the operator can operate without leaning forward even when there is a blind spot area in the movement range Rm.

An information output unit 20e that outputs information regarding the distance between the obstacle and the moving unit 40 may be provided. In this case, information about the distance can be notified internally or externally.

Next, the second to fourth embodiments of the present invention will be described. In the drawings and description of the second to fourth embodiments, the same or equivalent components and members as those of the first embodiment are designated by the same reference numerals. The description overlapping with the first embodiment will be omitted as appropriate, and the configuration different from that of the first embodiment will be mainly described.

[Second Embodiment]
A second embodiment of the present invention is a method for supporting the operation of a work machine. In this maneuvering support method, the step S71 for acquiring the image information Gp regarding the moving range Rm of the moving unit 40 of the work machine 100 having the moving unit 40 and the moving unit 40 avoiding contact with an obstacle based on the image information Gp. The steps S72 to S80 are included.

According to the configuration of the second embodiment, the same effects as those of the first embodiment are obtained.

[Third Embodiment]
A third embodiment of the present invention is a maintenance support method for the operation support system 1 of the work machine 100. The maintenance support method S90 includes a step S91 for determining the necessity of maintenance of the maneuvering support system 1 based on the comparison result between the acquired image information and the reference image information stored in advance.

The reference image information is the second image information Gs (hereinafter, image information Gs-A) acquired in a state where the boom 42, the arm 44, and the bucket 46 are arranged at the origin position in the past, for example, when the work machine 100 is introduced. ) May be. The acquired image information may be the second image information Gs (hereinafter referred to as image information Gs-B) acquired in a state where the boom 42, the arm 44, and the bucket 46 are arranged at the origin position at the present time. ..

In step S91, the past and present coordinates of the measurement points 14a to 14c are specified from the image information Gs-A and Gs-B, and when the difference exceeds the threshold value, the maneuvering support system 1 determines that maintenance is required. If the threshold value is not exceeded, it may be determined that maintenance is unnecessary.

When it is determined that maintenance is necessary, the information output unit 20e of the operation control unit 20 may output the determination result to the information terminal 50. Further, the information output unit 20e may output a predetermined alarm as sound or light based on the determination result.

According to the third embodiment, it is possible to accurately grasp the necessity of maintenance of the maneuvering support system 1. Further, the origin positions of the boom 42, the arm 44 and the bucket 46 can be calibrated with high accuracy.

[Fourth Embodiment]
A fourth embodiment of the present invention is a construction machine 1000. The construction machine 1000 operates the moving unit 40, the acquisition unit 10 that acquires the image information Gp regarding the moving range Rm of the moving unit 40, and the moving unit 40 to avoid contact with an obstacle based on the image information Gp. The operation control unit 20 is provided. The construction machine 1000 may be, for example, a machine that moves a bucket 46 to perform construction work. The moving portion 40 of the construction machine 1000 may be provided with various attachments such as a fork, a hammer, and a crusher instead of the bucket. According to the configuration of the fourth embodiment, the same effects as those of the first embodiment are obtained.

The examples of the embodiments of the present invention have been described in detail above. All of the above-described embodiments are merely specific examples for carrying out the present invention. The content of the embodiment does not limit the technical scope of the present invention, and many design changes such as modification, addition, and deletion of components are made within the scope of the invention as defined in the claims. It is possible. In the above-described embodiment, the contents that can be changed in such a design are described with notations such as "in the embodiment" and "in the embodiment", but the contents are designed without such notations. It's not that changes aren't tolerated.

[Modification example]
Hereinafter, a modified example will be described. In the drawings and description of the modified examples, the same or equivalent components and members as those in the embodiment are designated by the same reference numerals. The description overlapping with the embodiment will be omitted as appropriate, and the configuration different from that of the first embodiment will be mainly described.

In the description of the first embodiment, an example is shown in which the work machine is a construction machine for performing construction work by moving the bucket 46, but the present invention is not limited to this, and can be applied to work machines other than the construction machine. ..

In the description of the first embodiment, an example in which the obstacle is a person is shown, but the present invention is not limited thereto. The description of the first embodiment is also applicable to obstacles other than humans.

In the description of the first embodiment, an example in which the image sensor 12 is composed of two image sensors has been shown, but the present invention is not limited thereto. The image sensor 12 may be composed of a single image sensor or may be composed of three or more image sensors.

In the description of the first embodiment, an example in which the image sensor 12 is provided on the roof of the cockpit 38 is shown, but the present invention is not limited to this. For example, the image sensor 12 may include an external image sensor that captures an image from a position away from the work machine 100. For example, the external image sensor may be mounted on the drone, or may be hung on a wire or the like stretched in the sky. In this case, since an image sensor provided away from the work machine is used, it is possible to prevent omission of detection of obstacles.

In the description of the first embodiment, an example is shown in which the first to third distances D1 to D3 are constant regardless of the type of obstacle, but the present invention is not limited thereto. For example, the first to third distances D1 to D3 may be changed according to the type of obstacle. When the obstacle is a person, the first to third distances D1 to D3 may be made larger than in other cases.

In the description of the first embodiment, an example in which the information terminal 50 is provided in the cockpit 38 is shown, but the present invention is not limited to this. For example, the information terminal 50 may be a mobile terminal such as a tablet terminal owned by a worker outside the work machine 100.

In the description of the first embodiment, an example in which the work machine 100 is operated by an operator seated in the cockpit 32 is shown, but the present invention is not limited thereto. For example, the work machine may be autopilot or remote controlled.

The above-mentioned modification has the same action / effect as that of the first embodiment.

Any combination of the above-described embodiments and modifications is also useful as an embodiment of the present invention. The new embodiments resulting from the combination have the effects of the combined embodiments and variants.

1 ... Maneuvering support system, 8 ... People, 8b ... Foot 10 ... Acquisition unit, 12 ... Image sensor, 20 ... Motion control unit, 20c ... Obstacle identification unit , 20d ... distance identification unit, 20e ... information output unit, 20f ... avoidance control unit, 20g ... valve control unit, 20h ... person identification unit, 20j ... moving unit identification unit, 30 ... control support device, 32 ... driver's seat, 40 ... moving part, 100 ... work machine.

Claims (16)

An acquisition unit that acquires image information about the movement range of the moving unit of the work machine,
A maneuvering support system for a work machine including a motion control unit that causes the moving unit to operate to avoid contact with an obstacle based on the image information. The maneuvering support system according to claim 1, wherein the acquisition unit is configured to include image information about the moving unit in the image information. The maneuvering support system according to claim 1 or 2, wherein the acquisition unit includes a plurality of image sensors that image the moving unit from different positions. The maneuvering support system according to any one of claims 1 to 3, wherein the motion control unit controls the moving unit to reduce the moving speed of the moving unit when the moving unit approaches the obstacle within a first distance. The maneuvering support system according to any one of claims 1 to 4, wherein the motion control unit controls the moving unit to stop moving when the moving unit approaches the obstacle within a second distance. The maneuvering support system according to any one of claims 1 to 5, wherein the motion control unit controls the moving unit to change the moving course of the moving unit when the moving unit approaches within a third distance from the obstacle. The maneuvering support system according to any one of claims 1 to 6, wherein the motion control unit has an obstacle identification unit that identifies the type of the obstacle by using the feature information about the obstacle stored in advance. The maneuvering support system according to claim 7, further comprising a person identification unit that identifies the person's feet when the obstacle is a person and identifies the person's feet from the feet to a predetermined height as the person's range. The maneuvering support system according to claim 8, wherein the motion control unit includes the person identification unit. The maneuvering support system according to any one of claims 1 to 9, further comprising a moving unit identification unit that identifies a range of the moving unit using coordinate information about a predetermined portion stored in advance of the moving unit. The maneuvering support system according to claim 10, wherein the motion control unit includes the moving unit identification unit. The moving part is configured to move according to the maneuver of the operator.
The maneuvering support system according to any one of claims 1 to 10, wherein the moving unit forms a blind spot region in the moving range seen from the cockpit of the work machine. The maneuvering support system according to any one of claims 1 to 12, further comprising an information output unit that outputs information regarding the distance between the obstacle and the moving unit. Steps to acquire image information about the moving range of the moving part of the work machine,
A method for supporting the operation of a work machine, which includes a step of causing the moving portion to move to avoid contact with an obstacle based on the image information. A maintenance support method for a maneuvering support system, which comprises a step of determining the necessity of maintenance of the maneuvering support system according to claim 1, based on a comparison result between the acquired image information and a reference image information stored in advance. Moving part and
An acquisition unit that acquires image information regarding the movement range of the movement unit, and
A construction machine including a motion control unit that causes the moving unit to operate to avoid contact with an obstacle based on the image information.

Images