WO2022230592A1 - Maneuver support device and work machine - Google Patents

Abstract

Provided is a maneuver support device for supporting maneuvering of a work machine having a boom capable of rotating, moving up and down, and extending and contracting. The maneuver support device includes: a transparent display; a setting unit that sets a reference plane in a scene viewed through the display; and a control unit that displays a virtual image for supporting maneuvering of the work machine, on the display so as to be overlaid on the reference plane displayed on the display.

Description

Operation support device and work machine

The present invention relates to a work machine steering support device and a work machine.

Conventionally, when a load is transported by a work machine such as a mobile crane, the operator of the mobile crane has to operate the boom of the mobile crane by operating a turning operating tool, a hoisting operating tool, an extending/contracting operating tool, and a load. Operate a plurality of operating tools such as a winch operating tool for lifting or suspending the

In addition, the operator must operate the mobile crane while operating multiple operating tools at the same time so that the load does not come into contact with the structures at the work site. Therefore, the operator obtains information on the position of the load, information on the feature, etc. from the slinging worker or the like, thereby supplementing the information on the range and the time that the operator cannot visually recognize. In other words, the operator operates the crane based on the operator's visual information and the information from the outside instructor.

In addition, mobile cranes have a boom camera or the like to acquire information in areas that are difficult for the operator to recognize from the operator's seat. A mobile crane can acquire an image from above, such as a load being transported and surrounding buildings, by a boom camera. For example, it is as in Patent Document 1.

The crane described in Patent Document 1 acquires an image of a work site and an image of a load suspended on a hook with a boom camera provided at the tip of the boom, and a reference line centered on the load on the image. At least one of a scale, a directed line segment indicating the movement direction of the load, a trajectory of the load, etc. is superimposed and displayed. A crane operator operates a crane by comprehensively judging the visual information of the operator himself, the information from the instructor, and the image information of the boom camera.

JP 2019-156538 A

The crane operator cannot operate the crane at the operator's pace because he operates the crane according to the instructions given by the instructor and the timing of the instructions. In addition, it may take a long time to move the cargo if the operator cannot grasp the intention of the instructor, or if the content of the instruction from the instructor cannot be fully conveyed due to poor communication. On the other hand, the image from the boom camera is limited in the shooting direction and shooting range. Therefore, the crane operator cannot fully obtain the information necessary to efficiently move the load from the image from the boom camera.

An object of the present invention is to provide a steering support device and a working machine that can improve the working efficiency of the working machine.

One aspect of the steering assistance device according to the present invention is
A steering support device for assisting the operation of a work machine having a boom capable of turning, hoisting and telescoping,
a transparent display,
A setting section that sets a reference plane for the scenery seen through the display,
a control unit that displays a virtual image on the display for assisting the operation of the work machine so as to overlap with the reference plane displayed on the display.

According to the present invention, it is possible to provide a steering support device and a working machine that can improve the working efficiency of the working machine.

FIG. 1 is a perspective view of a transmission type display device attached to a user in a steering support device for a work machine according to an embodiment of the present invention. FIG. 2 is a perspective view of a transmissive display device in the steering support device for a working machine according to one embodiment of the present invention. FIG. 3 is a block diagram of a steering support device for a working machine according to one embodiment of the present invention. FIG. 4 is a schematic diagram of a virtual image displayed on a transmissive display device in the steering support device for a working machine according to one embodiment of the present invention. FIG. 5 is a schematic diagram of a state in which the reference plane of the virtual image displayed on the transmissive display device is changed in the steering support device for the working machine according to the embodiment of the present invention. FIG. 6 is a side view of a crane to which a work machine steering support device according to an embodiment of the present invention is connected. FIG. 7 is a control block diagram of a crane to which a work machine steering support device according to an embodiment of the present invention is connected. FIG. 8 is a block diagram when generating information necessary for information for displaying the first virtual image and the second virtual image by the steering support device for a working machine according to the embodiment of the present invention. FIG. 9 shows a state in which the steering support device for a working machine according to one embodiment of the present invention displays the first virtual image, the second virtual image, the third virtual image, the fifth virtual image, and the seventh virtual image. It is a schematic diagram. FIG. 10 is a block diagram for generating information required for information for displaying the third virtual image by the steering support device for a working machine according to one embodiment of the present invention. FIG. 11 is a block diagram for generating information required for information for displaying the fourth virtual image by the operation support device for work implements according to the embodiment of the present invention. FIG. 12 is a schematic diagram of a state in which the steering support device for working machines according to the embodiment of the present invention displays the third virtual image, the fourth virtual image, the fifth virtual image, and the seventh virtual image. FIG. 13 is a block diagram for generating information required for information for displaying the fifth virtual image by the steering assistance device for work implements according to one embodiment of the present invention. FIG. 14 is a block diagram for generating information required for information for displaying the sixth virtual image by the steering support device for work implements according to the embodiment of the present invention. FIG. 15 is a schematic diagram of a state in which the steering support device for a working machine according to one embodiment of the present invention displays the first virtual image, the second virtual image, the sixth virtual image, and the seventh virtual image. FIG. 16 is a block diagram for generating information necessary for information for displaying the seventh virtual image by the operation support device for work implements according to the embodiment of the present invention. FIG. 17 is a schematic diagram of a state in which the steering support device for working machines according to the embodiment of the present invention displays a modified example of the first virtual image.

A steering support device 1 for a work machine according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 5. FIG. FIG. 1 is a perspective view of a transmission type display device attached to a user in a steering support device for a work machine according to an embodiment of the present invention.

FIG. 2 is a perspective view of the transmissive display device 2 in the steering support device 1 for working equipment. FIG. 3 is a block diagram of the steering support device 1 for a working machine. FIG. 4 is a schematic diagram showing a virtual image Im displayed in the steering support device 1 for a working machine.

FIG. 5 is a schematic diagram in which the reference plane S1 for displaying the virtual image Im is changed in the steering support device 1 for the working machine. In this embodiment, a rough terrain crane (hereinafter simply referred to as "crane") will be described as a work machine.

However, the work machine may be any work machine provided with a boom that can be raised and lowered via a swivel base such as an all-terrain crane, a truck crane, and an aerial work platform.

As shown in FIGS. 1 and 2, the operation support device 1 for the crane 11 (see FIG. 6), which is a work machine, is a system that supports operation of the crane 11 using mixed reality. The steering support device 1 has a transmissive display device 2 , a communication device 8 and an image processing device 9 .

The transmissive display device 2 is a display device that allows the user to visually recognize the background of the transmissive display device 2 that overlaps the display screen of the transmissive display device 2 when viewed from the user. The transmissive display device 2 is a wearable terminal worn on the body by the user.

In this embodiment, the transmissive display device 2 will be described as a glasses-type wearable terminal. The transmissive display device 2 is attached to the operator of the crane 11 who is the user. The transmissive display device 2 includes a frame 3 , a transmissive display 4 , a GNSS receiver 5 for the display device, a camera 6 for the display device, and a control device 7 for the display device.

The frame 3 supports a transmissive display 4, a GNSS receiver 5 for display, a camera 6 for display, and a controller 7 for display. The frame 3 is configured as an eyeglass-type frame. The frame 3 is worn on the user's head. The frame 3 has a bridge supported by the user's nose and a pair of temples supported by both ears.

The transmissive display 4 is a display with a transmittance of approximately 50% or more. In other words, the transmissive display 4 is a transparent display that allows the user to see the scenery through the transmissive display 4 . The transmissive display 4 is composed of a liquid crystal display, an organic EL display, an inorganic EL display, or the like, which does not have a backlight, a polarizing filter and a color filter.

In other words, the transmissive display 4 does not have members such as a backlight and a filter that cover the back of the display screen facing the user. Therefore, the transmissive display 4 has a transparent background that overlaps with the transmissive display 4 when viewed from the user's viewpoint. Further, the transmissive display 4 can display a virtual image Im, which is an image generated by the function of the transmissive display 4, on the display screen.

In this specification, the virtual image is an image or video other than the landscape viewed by the user on the transmissive display device, and is a line, figure, image, video, or video image displayed by the transmissive display device. Dimensional image means 3D video. In other words, the virtual image is an image that can be displayed without discomfort in the scenery visually recognized by the user by converting an image of an object that does not actually exist into an image of a shape viewed from the user's viewpoint. The virtual image is displayed superimposed on the landscape viewed by the user on the transmissive display device. The virtual image is also an image for assisting the user in operating the work implement.

Also, in this specification, a viewpoint means a standing point when viewing an object. The user's viewpoint means the position of the user with respect to the object when the user is visually recognizing the object.

The transmissive display 4 transmits light that has entered from the back in a portion where the virtual image Im is not displayed. In addition, the transmissive display 4 blocks at least part of the light entering from the back in the portion where the virtual image Im is displayed.

Furthermore, the transmissive display 4 displays the virtual image Im in the light-shielded portion. At this time, a virtual image Im viewed from the user's viewpoint is displayed on the transmissive display 4 . Thereby, the transmissive display 4 allows the user to visually recognize the state in which the virtual image Im is superimposed on the landscape Ls viewed from the user's viewpoint.

The transmissive display 4 is provided on the frame 3. The transmissive display 4 is positioned so as to cover the viewing range of the right eye and the viewing range of the left eye of the user wearing the frame 3 . The transmissive display 4 is configured to be able to display the virtual image Im for the right eye and the virtual image Im for the left eye as parallax images on the display screen.

Thereby, the transmissive display 4 can display the stereoscopic virtual image Im in almost the entire visual field of the user. Incidentally, in the transmissive display device 2, a technique for displaying the virtual image Im seen from the user's viewpoint within the user's field of view is known.

The GNSS receiver 5 for the display device is a receiver that constitutes a global navigation satellite system. The GNSS receiver 5 for the display device receives ranging radio waves from satellites and detects latitude, longitude, altitude, azimuth, etc., which are the absolute coordinates of the GNSS receiver 5 for the display device.

The GNSS receiver 5 for the display device is provided on the frame 3 worn by the user. The display device GNSS receiver 5 detects information Ifa on the position of the user (operator).

The information Ifa on the user's position includes the latitude, longitude, altitude, azimuth, etc. of the transmissive display device 2 . That is, the information Ifa on the position of the user is information on the position of the transmissive display device 2 .

In this embodiment, the GNSS receiver 5 for the display device is provided in the frame 3, but it may be located at any position where the information Ifa regarding the position of the user can be detected. For example, the GNSS receiver 5 for display device may be attached to the body of the user wearing the transmissive display device 2 .

The display device camera 6 captures the line of sight of the user wearing the transmissive display device 2 . The display device camera 6 is provided on the frame 3 . The display device camera 6 is configured to photograph a range substantially equal to the user's visual field range from a position substantially equal to the user's viewpoint.

Although the display device camera 6 is provided in the frame 3 in the present embodiment, it may be located at a position where the user's visual field range can be photographed from the user's viewpoint.

As shown in FIG. 3, the display device control device 7 corresponds to an example of a control unit, and is a control device that causes the transmissive display 4 to display the virtual image Im. A CPU, a ROM, a RAM, an HDD, and the like are actually connected to the display device control device 7 via a bus. Alternatively, the display device control device 7 may be configured by a one-chip LSI or the like.

The display device control device 7 is connected to the transmissive display 4, the display device GNSS receiver 5, and the display device camera 6. The display device control device 7 is connected to the image processing device 9 via the communication device 8 . The display device control device 7 has information Ifd regarding the reference plane S1. In this embodiment, the reference plane S1 is set to the ground surface.

The display device control device 7 can acquire information Ifa on the position of the user from the display device GNSS receiver 5 . The display device control device 7 can acquire the image P captured by the display device camera 6 from the display device camera 6 .

Also, the display device control device 7 can acquire information If about the virtual image Im from the image processing device 9 . The display device control device 7 can generate the virtual image Im based on the acquired information If about the virtual image Im. Further, the display device control device 7 can cause the transmissive display 4 to display the virtual image Im.

In this specification, the information about the virtual image is the information necessary to display the virtual image on the display screen of the transmissive display device as an image on an arbitrary reference plane seen from the viewpoint of the user of the transmissive display device. means The information about the virtual image includes the coordinates of the virtual image, the display state, and the like.

As shown in FIGS. 3 and 4, the transmissive display device 2 can display a virtual image Im on the display screen of the transmissive display 4 while worn on the user's head. Further, the transmissive display device 2 can acquire the information Ifa on the user's position necessary for displaying the virtual image Im on the transmissive display 4 by the GNSS receiver 5 for the display device.

Based on the information If about the virtual image Im and the information Ifa about the position of the user, the transmissive display device 2 displays a virtual image viewed from the user's viewpoint on the display screen of the transmissive display 4 through which the scenery Ls is transmitted. Im can be generated.

Furthermore, the transmissive display device 2 can display the virtual image Im on the display screen of the transmissive display 4 . As a result, the transmissive display device 2 allows the user of the transmissive display device 2 to visually recognize the virtual image Im superimposed on the scenery Ls.

As shown in FIG. 3, the communication device 8 is a device that transmits and receives information between the transmissive display device 2, the image processing device 9, and the crane control device 30. The communication device 8 is provided in each of the display device control device 7 and the image processing device 9 of the transmissive display device 2 .

The communication device 8 transmits and receives information between the crane control device 30 and the image processing device 9 . Further, the communication device 8 transmits and receives information between the display device control device 7 and the image processing device 9 .

In this specification, a communication device means a device that transmits and receives information between an image processing device and a display device control device, an image processing device and an input device, and an image processing device and a work machine control device. . The communication device may be integrated with the display device control device and the image processing device.

The image processing device 9 is a control device that generates information If regarding the virtual image Im to be displayed on the transmissive display device 2 . The image processing device 9 is substantially connected with a CPU, a ROM, a RAM, an HDD, and the like via a bus. Alternatively, the image processing device 9 may be configured by a one-chip LSI or the like.

In this embodiment, the image processing device 9 is provided in the transmissive display device 2 . The image processing device 9 stores various programs and data for controlling operations of the transmissive display device 2 and the like and processing image data. The image processing device 9 may be configured integrally with the display device control device 7 or the crane control device 30 .

The image processing device 9 is connected to the display device control device 7 of the transmissive display device 2 via the communication device 8 . The image processing device 9 acquires the information Ifa on the position of the user detected by the GNSS receiver 5 for the display device from the control device 7 for the display device via the communication device 8 . Further, the image processing device 9 acquires the image P captured by the display device camera 6 from the display device control device 7 via the communication device 8 .

The image processing device 9 also acquires information Ifb regarding the position of the crane 11 from the crane control device 30 via the communication device 8 . The information Ifb regarding the position of the crane 11 includes the latitude, longitude, altitude, azimuth of the vehicle, etc. at which the turning center of the crane 11 is located. The image processing device 9 also transmits information Ife regarding the target position Lt of the main hook 20 a to the crane control device 30 via the communication device 8 .

The image processing device 9 also acquires information Ifc regarding the attitude of the crane 11 from the crane control device 30 via the communication device 8 . The information Ifc about the posture of the crane 11 includes the swing angle of the crane 11, the boom hoisting angle, the length of the boom, the payout length of the main wire rope 24, the payout length of the sub wire rope 26, and the operating tools of the crane 11. Includes direction and amount of operation. That is, the information Ifc about the posture of the crane 11 includes information about the current position Lp of the main hook 20a of the crane 11, which is a reference member, for example.

Also, the image processing device 9 acquires information Ifd regarding the reference plane S1 from the display device control device 7 via the communication device 8 . The information Ifd about the reference plane S1 includes the position and inclination of the reference plane S1 (see FIG. 4) that serves as a reference for the position of displaying the virtual image Im viewed from the user's viewpoint.

The image processing device 9 calculates a plurality of reference coordinates C based on the obtained information Ifc regarding the posture of the crane 11 and information Ifd regarding the reference plane S1. The reference coordinates C are the coordinates of the virtual image Im displayed on the reference plane S1. The image processing device 9 corresponds to an example of a setting unit that sets the reference plane S<b>1 to the scenery seen through the transmissive display 4 .

The image processing device 9 generates information If about the virtual image Im based on the obtained information Ifa about the position of the user, information Ifb about the position of the crane 11, and information Ifd about the reference plane S1. The information If about the virtual image Im is information for displaying a plurality of reference coordinates C on the display screen of the transmissive display 4 as a virtual image Im on the reference plane S1 viewed from the user's viewpoint. The image processing device 9 transmits information If about the generated virtual image Im of the crane to the display device control device 7 .

As shown in FIGS. 3 and 4, the transmissive display device 2 acquires information If regarding the virtual image Im of the crane via the communication device 8. FIG. When the reference plane S1 is the ground surface, a virtual image Im viewed from the user's viewpoint is displayed on the display screen of the transmissive display 4 overlapping the ground surface viewed by the user through the transmissive display 4. do.

As shown in FIG. 5, when the image processing device 9 detects the motion of a part of the user's body or the motion of a specific marker from the obtained image P captured by the display device camera 6, the image processing device 9 detects information about the virtual image Im. It is recognized as an input action of inputting If.

For example, when the image processing device 9 detects that the user's hand grasps and moves the reference plane S1 viewed from the user's viewpoint, the image processing device 9 moves the reference plane S1 to a new position and position according to the direction and amount of movement of the hand. Change to a new reference plane S2 having an inclination.

Similarly, for example, when the image processing device 9 detects that the user's hand indicates an arbitrary position viewed from the user's viewpoint, the image processing device 9 detects the position where the virtual image Im is displayed according to the direction and amount of movement of the hand. to change Thus, the display device camera 6 has a function as an input device for inputting the information If regarding the virtual image Im.

In this specification, an input device means a device that inputs information for displaying a virtual image to the image processing device. The input device may be configured integrally with the image processing device and the transmissive display device. The input device may be an input by movement of a part of the user's body or movement of a marker within the visual field of the user wearing the transmissive display device.

The steering support device 1 configured in this manner acquires information Ifc regarding the attitude of the crane 11 from the crane control device 30 via the communication device 8 . Further, the steering assistance device 1 acquires information Ifa on the user's position and information Ifd on the reference plane S1 via the communication device 8 .

The steering assistance device 1 generates information If regarding the virtual image Im based on each piece of information acquired by the image processing device 9 . The steering support device 1 causes the transmissive display device 2 to display the virtual image Im based on the information If regarding the virtual image Im. The steering support device 1 displays a virtual image Im representing various information about the crane 11 superimposed on the landscape Ls viewed by the user, thereby suppressing the influence of the information acquired from the instructor on the steering. , to support the operation of the crane 11 by the user so that the crane 11 can be operated efficiently.

A crane 11, which is a working machine according to an embodiment of the present invention, will be described below with reference to FIGS. 6 and 7. FIG. FIG. 6 shows the overall configuration of a crane 11 to which the steering support device 1 is connected. FIG. 7 shows the control configuration of the crane 11 to which the steering support device 1 is connected. In this embodiment, a rough terrain crane will be described, but any working machine such as an all-terrain crane, a truck crane, or an aerial work platform provided with a boom that can be raised and lowered via a swivel platform may be used.

　As shown in Figures 6 and 7, the crane 11 is a mobile crane that can be moved to any location. The crane 11 has a vehicle 12, a crane device 16, a boom camera 19b as a photographing device, and a crane control device 30 (see FIG. 7).

The vehicle 12 is a traveling body that transports the crane device 16. Vehicle 12 has a plurality of wheels 13 . The vehicle 12 runs using the engine 14 as a power source. Vehicle 12 has outriggers 15 . The outriggers 15 have overhang beams that can be hydraulically extended on both sides in the width direction of the vehicle 12 and hydraulic jack cylinders that can be extended in a direction perpendicular to the ground.

The crane device 16 is a working device that lifts the load W with a wire rope. The crane device 16 includes a swivel base 17, a boom 19, a main hook block 20, a sub-hook block 21, a hoisting hydraulic cylinder 22, a main winch 23, a main wire rope 24, a sub-winch 25, a sub-wire rope 26, a cabin 27, and the like. equip.

The swivel base 17 is a rotating device that allows the crane device 16 to swivel. The swivel base 17 is provided on the frame of the vehicle 12 via an annular bearing. The swivel base 17 is rotatable around the center of an annular bearing.

The swivel base 17 has a hydraulic swivel hydraulic motor 17a, which is an actuator. The swivel base 17 is configured to be swivelable in one direction and the other direction by a swivel hydraulic motor 17a.

The crane GNSS receiver 18 (see FIG. 7) is a receiver that constitutes the Global Navigation Satellite System.

The crane GNSS receiver 18 receives ranging radio waves from satellites and detects the absolute coordinates of the crane GNSS receiver 18 such as latitude, longitude, altitude and azimuth. A crane GNSS receiver 18 is provided on the swivel base 17 .

The crane GNSS receiver 18 detects information Ifb regarding the position of the crane 11 . The information Ifb regarding the position of the crane 11 includes the latitude, longitude, altitude, azimuth, and the like of the crane 11 .

The boom 19 is a movable strut that supports the main wire rope 24 and the sub wire rope 26. The boom 19 is composed of a plurality of boom members. The base end of the base boom member of the boom 19 is swingably provided substantially at the center of the swivel base 17 .

The boom 19 has telescoping hydraulic cylinders 19a (see FIG. 7), which are actuators for telescoping each boom member, and hoisting hydraulic cylinders 22 for hoisting the boom members. The boom 19 is axially extended and retracted by a telescopic hydraulic cylinder 19a.

Also, the boom 19 has a boom camera 19b (see FIG. 7) which is a photographing device. Furthermore, the boom 19 has a jib 19c which is an extension member. The image captured by the boom camera 19b is displayed on the display device of the crane 11 or the like.

The main hook block 20 and the sub hook block 21 are parts for hanging the load W. The main hook block 20 has a plurality of hook sheaves around which the main wire rope 24 is wound, and a main hook 20a for hanging the load W. The sub-hook block 21 has a sub-hook 21a for hanging the load W.

The hoisting hydraulic cylinder 22 is an actuator that raises and lowers the boom 19 and holds the attitude of the boom 19 . The end of the cylinder portion of the hoisting hydraulic cylinder 22 is connected to the swivel base 17 so as to be hoistable. The end of the rod portion of the hoisting hydraulic cylinder 22 is pivotably connected to the base boom member of the boom 19 .

The main winch 23 is a device that carries in (hoists) and lets out (lowers) the main wire rope 24 . The sub winch 25 is a device that carries in (winds up) and lets out (winds down) the sub wire rope 26 .

The main winch 23 is rotated by a main hydraulic motor (not shown) whose actuator is a main drum around which the main wire rope 24 is wound. The sub winch 25 is rotated by a sub hydraulic motor (not shown) whose actuator is a sub drum around which the sub wire rope 26 is wound.

The cabin 27 is a housing that covers the cockpit. The cabin 27 is mounted on the swivel base 17 . The cabin 27 is provided with a cockpit (not shown). At the operator's seat, there are a traveling operating tool 28 for operating the vehicle 12 to travel, a turning operating tool 29a for operating the turning hydraulic motor 17a of the crane device 16, a telescopic operating tool 29b for operating a telescopic cylinder, and a hoisting/raising operation tool. A crane device operating tool 29 including a hoisting operating tool 29c for operating the hydraulic cylinder 22, a main operating tool 29d for operating the main winch 23, and a sub operating tool 29e for operating the sub winch 25 are provided. (See Figure 2).

As shown in FIG. 7 , the crane control device 30 controls each actuator of the crane 11 . The crane control device 30 is provided inside the cabin 27 . In the crane control device 30, a CPU, a ROM, a RAM, an HDD, and the like are connected via a bus. The crane control device 30 may be composed of a one-chip LSI or the like.

The crane control device 30 stores various programs and data for controlling the operation of each actuator, switching valve, etc., and for processing image data. The crane control device 30 has a communication device 30a. The communication device 30 a transmits and receives information to and from the image processing device 9 of the steering support device 1 .

The crane controller 30 is connected to the crane GNSS receiver 18. The crane control device 30 can acquire information Ifb regarding the position of the crane 11 . The crane control device 30 is connected to the travel operating tool 28 and the crane device operating tool 29 .

The crane control device 30 can acquire operation signals of each of the traveling operation tool 28 and the crane device operation tool 29 . The crane control device 30 generates a control signal for the vehicle 12 based on an operation signal generated by operating the traveling operation tool 28 .

Similarly, the crane control device 30 generates a control signal for the crane device 16 based on an operation signal generated by operating the crane device operating tool 29 . The crane controller 30 can transmit the generated control signals to each actuator of the crane apparatus 16 .

The crane control device 30 is connected to the image processing device 9 of the steering support device 1 via a communication device 30a. The crane control device 30 transmits to the image processing device 9 information Ifb regarding the position of the crane 11, information Ifc regarding the posture of the crane 11, and information Ifh regarding the path of the main hook 20a. The crane control device 30 can acquire information Ife regarding the target position Lt of the main hook 20a from the image processing device 9 via the communication device 30a.

The crane 11 configured in this way can move the vehicle 12 to an arbitrary position by operating the operating tool 28 for traveling. Moreover, the crane 11 can convey the load W to an arbitrary position by turning, raising and lowering, and extending and contracting the boom 19 by operating the operating tool 29 for the crane device. Further, the crane 11 can lift or suspend the load W with the main winch 23 or the like by operating the crane device operation tool 29 .

Next, the operation assistance provided by the operation assistance device 1 to the operator of the crane 11 will be described with reference to FIGS. 8 to 16. FIG.

FIG. 8 shows the acquisition state of information necessary for information for the operation support device 1 of the crane 11 to display the first virtual image Im1 and the second virtual image Im2. FIG. 9 shows a state in which the steering support device 1 of the crane 11 displays a first virtual image Im1, a second virtual image Im2, a third virtual image Im3, a fifth virtual image Im5 and a seventh virtual image Im7.

FIG. 10 shows the acquisition state of information necessary for information for the operation support device 1 of the crane 11 to display the third virtual image Im3. FIG. 11 shows the acquisition state of information necessary for the information for the operation support device 1 of the crane 11 to display the fourth virtual image Im4.

FIG. 12 shows a state in which the steering support device 1 of the crane 11 displays the third virtual image Im3, the fourth virtual image Im4, the fifth virtual image Im5 and the seventh virtual image Im7. FIG. 13 shows the acquisition state of information necessary for the information for the steering support device 1 of the crane 11 to display the fifth virtual image Im5.

FIG. 14 shows the acquisition state of information necessary for the information for the operation support device 1 of the crane 11 to display the sixth virtual image Im6. FIG. 15 shows a state in which the steering support device 1 of the crane 11 displays the first virtual image Im1, the second virtual image Im2, the sixth virtual image Im6 and the seventh virtual image Im7. FIG. 16 shows the acquisition state of information necessary for the information for the steering support device 1 of the crane 11 to display the seventh virtual image Im7.

The operator of the crane 11 is the user of the transmissive display device 2. An operator of the crane 11 operates the crane 11 with the transmissive display device 2 mounted on the head. Also, the operator of the crane 11 operates the crane 11 while seated in the cabin 27 (see FIG. 6) of the crane 11 . In this embodiment, the main hook 20a of the crane 11 is used as the reference member. Note that the reference member may be the load W conveyed by the crane 11 .

<Track Display at Current Position Lp of Main Hook 20a>
As shown in FIGS. 8 and 9, the steering support device 1 displays the trajectory of the main hook 20a, which is the reference member, at the current position Lp on the transmissive display device 2 as a first virtual image Im1.

In this specification, the trajectory means a figure representing a set of coordinates through which the reference member of the work machine passes by the operation of each operation tool of the work machine with a continuous line. Further, the trajectory includes a figure represented by a continuous line obtained by transforming a set of coordinates through which the reference member of the working machine passes by operation of each operating tool into coordinates on an arbitrary reference plane.

Based on the obtained information Ifc about the attitude of the crane 11 and information Ifd about the reference plane S1, the image processing device 9 determines the current position Lp of the main hook 20a with reference to the tip of the boom 19 and the position of the crane 11 for the crane device. The trajectory from the current position Lp of the main hook 20a moved by the crane 11 when the operating tool 29 (see FIG. 7) is operated (hereinafter simply referred to as "the trajectory from the current position Lp of the main hook 20a"). on the reference plane S1 to calculate a plurality of first reference coordinates C1.

Based on the acquired information Ifa on the position of the operator (user) and information Ifb on the position of the crane 11, the image processing device 9 defines the plurality of first reference coordinates C1 on the reference plane S1 when viewed from the operator's viewpoint. Information to be displayed on the display screen of the transmissive display 4 as the upper first virtual image Im1 is generated.

That is, the image processing device 9 converts information about the current position Lp of the main hook 20a on the reference plane S1 viewed from the operator's viewpoint and the trajectory of the main hook 20a from the current position Lp into the first virtual image of the crane 11. It is generated as information If1 related to Im1. The image processing device 9 transmits information If1 regarding the generated first virtual image Im1 of the crane 11 to the display device control device 7 .

The display device control device 7 acquires information If1 regarding the first virtual image Im1 of the crane 11 via the communication device 8. Based on the information If1 on the first virtual image Im1 of the crane 11, the display device control device 7 displays the first virtual image Im1 on the display screen of the transmissive display 4 on the reference plane S1 viewed from the operator's viewpoint. The current position Lp of the main hook 20a and the trajectory from the current position Lp of the main hook 20a are displayed.

When the reference plane S1 is the ground surface, the display device control device 7 displays an image on the display screen of the transmissive display 4 that overlaps the ground surface viewed by the operator through the transmissive display 4, as viewed from the operator's viewpoint. The current position Lp of the main hook 20a on the ground surface and the trajectory of the main hook 20a from the current position Lp are displayed as a first virtual image Im1.

The first virtual image Im1 projects the trajectory along which the main hook 20a moves when the operator operates the turning operation tool 29a from the state where the main hook 20a is located at the current position Lp onto the reference plane S1. It can be regarded as an image with

The first virtual image Im1 shows the trajectory along which the main hook 20a moves when the operator operates the telescoping operation tool 29b or the hoisting operation tool 29c from the state where the main hook 20a is located at the current position Lp. It may be an image projected onto S1.

Alternatively, the first virtual image Im1 can be obtained by performing at least two operations selected by the operator from the turning operating tool 29a, the extending/contracting operating tool 29b, and the hoisting operating tool 29c from the state where the main hook 20a is located at the current position Lp. It may be an image in which the trajectory along which the main hook 20a moves when the tools are operated simultaneously is projected onto the reference plane S1.

The operator uses the transmissive display device 2 to superimpose the current position Lp of the main hook 20a and the trajectory from the current position Lp of the main hook 20a on the ground surface of the work site, which is the actual scenery Ls. to recognize.

The operator can visually recognize the current position Lp of the main hook 20a and the trajectory from the current position Lp of the main hook 20a on the ground surface even when the main hook 20a cannot be viewed from the cockpit.

By superimposing and displaying the first virtual image Im1 within the field of view of the operator, the steering support device 1 is used to grasp the current position Lp of the main hook 20a and the positional relationship between the main hook 20a and the feature. Provide information to pilots.

Further, the steering support device 1 displays the trajectory of the main hook 20a from any direction and position desired by the operator within the field of view of the operator. Therefore, even if the instruction from the instructor in charge of slinging is insufficient, the operator supplements the information necessary for operating the crane 11 by the trajectory of the main hook 20a displayed within the operator's field of view. be able to. As a result, the operator can suppress the influence of the information acquired from the instructor on the operation and operate the crane 11 efficiently.

<Track display at target position Lt of main hook 20a>
As shown in FIGS. 8 and 9, the steering support device 1 displays the trajectory of the main hook 20a at the target position Lt with the tip of the boom 19 as a reference on the transmissive display device 2 as a second virtual image Im2. good too.

The image processing device 9 acquires an image of the operator's hand or marker from the operator's viewpoint using the display device camera 6 having the function of an input device. The image processing device 9 determines whether or not the information Ife regarding the target position Lt of the main hook 20a is based on the motion of the operator's hand or the type of marker in the acquired image.

When the image processing device 9 determines that the acquired image is the information Ife regarding the target position Lt of the main hook 20a, the shape, position and movement direction of the operator's hand or the type, position and movement direction of the marker in the image, etc. to specify the target position Lt of the main hook 20a.

The image processing device 9 calculates a plurality of second reference coordinates C2 based on the obtained information Ifc regarding the posture of the crane 11, information Ifd regarding the reference plane S1, and information Ife regarding the target position Lt of the main hook 20a.

The plurality of second reference coordinates C2 are the target position Lt of the main hook 20a and the target position of the main hook 20a to be moved by the crane 11 when the crane device operating tool 29 (see FIG. 7) of the crane 11 is operated. These are coordinates for displaying the trajectory from Lt (hereinafter simply referred to as "the trajectory from the target position Lt of the main hook 20a") on the reference plane S1.

Based on the acquired information Ifa on the position of the operator and information Ifb on the position of the crane 11, the image processing device 9 calculates a plurality of second reference coordinates C2 on the reference plane S1 viewed from the viewpoint of the operator. Information to be displayed on the display screen of the transmissive display 4 as the virtual image Im2 is generated.

That is, the image processing device 9 converts information about the target position Lt of the main hook 20a on the reference plane S1 viewed from the operator's viewpoint and the trajectory of the main hook 20a from the target position Lt into the second virtual image of the crane 11. It is generated as information If2 related to Im2.

The image processing device 9 transmits information If2 regarding the generated second virtual image Im2 of the crane 11 to the display device control device 7 .

The display device control device 7 acquires information If2 regarding the second virtual image Im2 of the crane 11 via the communication device 8. The display device control device 7 displays the second virtual image Im2 on the display screen of the transmissive display 4 based on the information If2 regarding the second virtual image Im2 of the crane 11 . Here, the second virtual image Im2 is an image relating to the target position Lt of the main hook 20a on the reference plane S1 and the trajectory of the main hook 20a from the target position Lt as seen from the operator's viewpoint.

The operator uses the transmissive display device 2 to display the target position Lt of the main hook 20a and the trajectory from the target position Lt of the main hook 20a superimposed on the ground surface of the work site, which is the actual scenery Ls. recognize as

When both the trajectory of the main hook 20a from the current position Lp, which is the first virtual image Im1, and the trajectory of the main hook 20a from the target position Lt, which is the second virtual image Im2, are displayed together, the operator is placed in the cockpit Therefore, even when the main hook 20a cannot be visually recognized, the positional relationship between the current position Lp and the target position Lt of the main hook 20a and the positional relationship between the main hook 20a and the feature at the target position Lt can be visually recognized on the ground surface.

Therefore, even if the instruction from the instructor is insufficient, the operator moves the main hook 20a to the target position Lt by following the trajectory of the main hook 20a at the target position Lt displayed in the operator's field of view. can supplement the necessary information for As a result, the operator can suppress the influence of the information acquired from the instructor on the operation and efficiently operate the work machine.

<Perpendicular display at current position Lp of main hook 20a>
As shown in FIGS. 9 and 10, the steering assistance device 1 uses a transmissive image Im3, which is a vertical line passing through the current position Lp or the display position of the main hook 20a (that is, a line parallel to the vertical direction) as a third virtual image Im3. It may be displayed on the display device 2 .

Based on the obtained information Ifc about the posture of the crane 11 and information Ifd about the reference plane S1, the image processing device 9 converts the current position Lp of the main hook 20a and the trajectory of the main hook 20a from the current position Lp into the reference plane. A plurality of first reference coordinates C1 are calculated for display on S1.

Further, the image processing device 9 calculates a plurality of third reference coordinates C3 for displaying a perpendicular line passing through the current position Lp of the main hook 20a based on the calculated first reference coordinates C1.

Based on the obtained information Ifa about the position of the operator and information Ifb about the position of the crane 11, the image processing device 9 transmits the plurality of third reference coordinates C3 as a third virtual image Im3 viewed from the viewpoint of the operator. Information to be displayed on the display screen of the mold display 4 is generated.

That is, the image processing device 9 generates information about a perpendicular line passing through the current position Lp of the main hook 20a seen from the operator's viewpoint as information If3 about the third virtual image Im3 of the crane 11.

The image processing device 9 transmits information If3 regarding the generated third virtual image Im3 of the crane 11 to the display device control device 7 . The information If3 regarding the third virtual image Im3 of the crane 11 may include information regarding a perpendicular line passing through the target position Lt of the main hook 20a.

The display device control device 7 acquires information If3 regarding the third virtual image Im3 of the crane 11 via the communication device 8 . The display device control device 7 displays the third virtual image Im3 on the display screen of the transmissive display 4 based on the information If3 regarding the third virtual image Im3 of the crane 11 . Here, the third virtual image Im3 is an image relating to a vertical line passing through the current position Lp of the main hook 20a as seen from the operator's viewpoint.

The operator perceives the transmissive display device 2 as if a vertical line indicating the current position Lp of the main hook 20a is superimposed on the actual scenery Ls.

When both the vertical line from the current position Lp of the main hook 20a, which is the third virtual image Im3, and the vertical line from the target position Lt of the main hook 20a, which is the second virtual image Im2, are displayed together, the operator is in the cockpit Therefore, the positional relationship between the current position Lp and the target position Lt of the main hook 20a can be recognized even when the main hook 20a cannot be visually recognized.

Also, the operator can visually recognize the positional relationship between the vertical trajectory of the main hook 20a at the current position Lp or the target position Lt of the main hook 20a and the feature. Therefore, even if the instruction from the instructor is insufficient, the operator can suppress the influence of the information acquired from the instructor on the operation and operate the crane 11 efficiently.

<Track Display of Main Hook 20a on New Reference Plane S2>
As shown in FIGS. 11 and 12, the steering assistance device 1 converts at least one of the trajectory of the main hook 20a at the current position Lp and the trajectory at the target position Lt on an arbitrarily designated new reference plane S2 to a fourth It may be displayed on the transmissive display device 2 as a virtual image Im4.

The image processing device 9 acquires an image P of the operator's hand or marker from the operator's viewpoint using the display device camera 6 having the function of an input device. The image processing device 9 determines whether or not the information Iff regarding the new reference plane S2 from the motion of the operator's hand or the type of the marker in the acquired image P.

When the image processing device 9 determines that the acquired image is the information Iff regarding the new reference plane S2, the image processing device 9 determines from the operator's hand shape, position and movement direction in the image P or the type, position and movement direction of the marker, etc. The position and inclination of the new reference plane S2 are specified.

The image processing device 9 calculates a plurality of fourth reference coordinates C4 based on the information Ifc regarding the posture of the crane 11 and the information Iff regarding the new reference plane S2. The fourth reference coordinates C4 are coordinates for displaying the current position Lp of the main hook 20a and the trajectory of the main hook 20a from the current position Lp on the new reference plane S2.

Based on the acquired information Ifa on the position of the operator and information Ifb on the position of the crane 11, the image processing device 9 plots the plurality of fourth reference coordinates C4 on the new reference plane S2 viewed from the viewpoint of the operator. Information to be displayed on the display screen of the transmissive display 4 as the fourth virtual image Im4 is generated.

That is, the image processing device 9 transmits information about the current position Lp of the main hook 20a on the new reference plane S2 viewed from the operator's viewpoint and the trajectory of the main hook 20a from the current position Lp to the fourth It is generated as information If4 related to the virtual image Im4.

The image processing device 9 transmits information If4 regarding the generated fourth virtual image Im4 of the crane 11 to the display device control device 7 .

The information If4 regarding the fourth virtual image Im4 of the crane 11 may include information regarding the target position Lt of the main hook 20a and the trajectory of the main hook 20a from the target position Lt.

The display device control device 7 acquires information If4 regarding the fourth virtual image Im4 of the crane 11 via the communication device 8. The display device control device 7 displays the fourth virtual image Im4 on the display screen of the transmissive display 4 based on the information If4 regarding the fourth virtual image Im4 of the crane 11 . Here, the fourth virtual image Im4 is an image regarding the current position Lp of the main hook 20a on the new reference plane S2 and the trajectory of the main hook 20a from the current position Lp as viewed from the operator's viewpoint.

The operator uses the transmissive display device 2 to display the current position Lp of the main hook 20a and the trajectory from the current position Lp of the main hook 20a on a new reference plane S2 arbitrarily set in the work site, which is the actual scenery Ls. are superimposed and displayed.

When both the trajectory of the main hook 20a from the current position Lp and the trajectory of the main hook 20a from the target position Lt are displayed as the fourth virtual image Im4, the operator cannot visually recognize the main hook 20a from the cockpit. However, the positional relationship between the current position Lp and the target position Lt of the main hook 20a and the positional relationship between the main hook 20a and the feature at the target position Lt can be visually recognized on the new reference plane S2.

Therefore, even if the instruction from the instructor is insufficient, the operator can follow the trajectory at the current position Lp and the target position Lt of the main hook 20a on the new reference plane S2 displayed in the operator's field of view. Information necessary for moving the main hook 20a to the target position Lt can be supplemented. As a result, the operator can suppress the influence of the information acquired from the instructor on the operation and operate the crane 11 efficiently.

<Display of movement direction of main hook 20a>
As shown in FIGS. 12 and 13, the steering support device 1 may display a vector diagram indicating the moving direction of the main hook 20a on the transmissive display device 2 as the fifth virtual image Im5.

The image processing device 9 generates information If1 regarding the first virtual image Im1 of the crane 11 as described above. Further, the image processing device 9 acquires information Ifg about the moving direction and moving speed of the main hook 20 a from the crane control device 30 via the communication device 8 .

The image processing device 9 calculates a plurality of fifth reference coordinates C5 based on the information Ifc regarding the attitude of the crane 11, the information Ifd regarding the reference plane S1, and the information Ifg regarding the moving direction and moving speed of the main hook 20a. The fifth reference coordinates C5 are coordinates for displaying a vector diagram representing the moving direction and moving speed of the main hook 20a on the reference plane S1.

Based on the obtained information Ifa about the position of the operator and the information Ifb about the position of the crane 11, the image processing device 9 transmits a plurality of fifth reference coordinates C5 as a fifth virtual image Im5 viewed from the viewpoint of the operator. Information to be displayed on the display screen of the mold display 4 is generated.

That is, the image processing device 9 generates information about a vector diagram indicating the moving direction of the main hook 20a from the current position Lp as seen from the operator's viewpoint as information If5 about the fifth virtual image Im5 of the crane 11.

The image processing device 9 transmits the generated information If1 regarding the first virtual image Im1 of the crane 11 and the information If5 regarding the fifth virtual image Im5 regarding the crane 11 to the display device control device 7 .

The display device control device 7 acquires information If1 regarding the first virtual image Im1 of the crane 11 and information If5 regarding the fifth virtual image Im5 of the crane 11 via the communication device 8.

The display device control device 7 displays the current position Lp of the main hook 20a seen from the operator's viewpoint, which is the first virtual image Im1, on the transmissive display 4 based on the information If1 about the first virtual image Im1 about the crane 11. Display on the display screen.

In addition, based on the information If5 on the fifth virtual image Im5 of the crane 11, the display device control device 7 determines the moving direction of the main hook 20a from the current position Lp as seen from the operator's viewpoint, which is the fifth virtual image Im5. is displayed on the display screen of the transmissive display 4 .

The operator perceives the transmissive display device 2 as if the moving direction of the main hook 20a from the current position Lp by operating the crane device operation tool 29 is superimposed on the actual scenery Ls.

When both the current position Lp of the main hook 20a, which is the first virtual image Im1, and the target position Lt of the main hook 20a, which is the second virtual image Im2, are displayed, the operator can visually recognize the main hook 20a from the cockpit. Even in a state where it is not possible, it is possible to recognize the positional relationship between the moving direction of the main hook 20a from the current position Lp and the target position Lt by operating the operating tool 29 for crane device. Therefore, even if the instruction from the instructor is insufficient, the operator can suppress the influence of the information acquired from the instructor on the operation and operate the crane 11 efficiently.

<Route display of the main hook 20a>
As shown in FIGS. 14 and 15, the steering support device 1 may display the route from the current position Lp of the main hook 20a to the target position Lt on the transmissive display device 2 as a sixth virtual image Im6.

The image processing device 9 acquires information Ifc regarding the attitude of the crane 11 including information regarding the current position Lp of the main hook 20a. Furthermore, the image processing device 9 acquires information Ife on the target position Lt of the main hook 20a by the display device camera 6 . The image processing device 9 transmits information Ife regarding the target position Lt of the main hook 20 a to the crane control device 30 via the communication device 8 .

The crane control device 30 acquires information Ife regarding the target position Lt of the main hook 20a. The crane control device 30 calculates the path of the main hook 20a that minimizes the pump flow rate of each actuator when the main hook 20a is moved from the current position Lp to the target position Lt of the main hook 20a.

The crane control device 30 transmits information Ifh regarding the route of the main hook 20 a calculated via the communication device 8 to the image processing device 9 . The method of calculating the path that minimizes the pump flow rate of each actuator when the main hook 20a is moved from the current position Lp to the target position Lt is a known technique.

The image processing device 9 generates information If1 regarding the first virtual image Im1 of the crane 11 and information If2 regarding the second virtual image Im2 of the crane 11 as described above.

Further, the image processing device 9 acquires information Ifh regarding the route of the main hook 20a. The image processing device 9 calculates a plurality of sixth reference coordinates C6 based on the obtained information Ifc regarding the posture of the crane 11 and information Ifh regarding the path of the main hook 20a. The sixth reference coordinates C6 are coordinates for displaying the route from the current position Lp of the main hook 20a to the target position Lt.

Based on the obtained information Ifa about the position of the operator and information Ifb about the position of the crane 11, the image processing device 9 transmits a plurality of sixth reference coordinates C6 as a sixth virtual image Im6 seen from the viewpoint of the operator. Information to be displayed on the display screen of the mold display 4 is generated.

That is, the image processing device 9 generates information about the route from the current position Lp of the main hook 20a to the target position Lt seen from the operator's viewpoint as information If6 about the sixth virtual image Im6 of the crane 11.

The image processing device 9 displays the generated information If1 about the first virtual image Im1 about the crane 11, the information If2 about the second virtual image Im2 about the crane 11, and the information If6 about the sixth virtual image Im6 about the crane 11. is transmitted to the control device 7 for use.

The display device control device 7 transmits information If1 about the first virtual image Im1 about the crane 11, information If2 about the second virtual image Im2 about the crane 11, and information If2 about the sixth virtual image Im6 about the crane 11 via the communication device 8. Acquire information If6.

The display device control device 7 displays the first virtual image Im1 and the second virtual image Im2 on the display screen of the transmissive display 4 as well as the sixth virtual image Im6 of the crane 11 based on the information If6 about the sixth virtual image Im6. display. Here, the sixth virtual image Im6 is an image regarding the route from the current position Lp of the main hook 20a to the target position Lt as seen from the operator's viewpoint.

The operator recognizes that the route from the current position Lp of the main hook 20a to the target position Lt is superimposed and displayed on the ground surface of the work site, which is the actual scenery Ls, by the transmissive display device 2. .

The operator can visually recognize in advance the positional relationship between the current position Lp of the main hook 20a and the target position Lt, and the positional relationship between the route of the main hook 20a and the feature. Therefore, the operator moves the main hook 20a from the current position Lp to the target position Lt along the route of the main hook 20a displayed in the operator's visual field even if the instruction from the instructor is insufficient. can supplement the necessary information for As a result, the operator can suppress the influence of the information acquired from the instructor on the operation and operate the crane 11 efficiently.

<Indicator display>
As shown in FIGS. 15 and 16, the maneuvering support device 1 may display various indicators related to the maneuvering of the crane 11 on the transmissive display device 2 as a seventh virtual image Im7.

The image processing device 9 has information Ifi regarding a specific area A of the crane 11 that displays only a specific virtual image Im from the viewpoint of the operator. The image processing device 9 acquires information Ifj regarding the operating state of the crane 11 from the crane control device 30 via the communication device 8 .

The image processing device 9, based on the information Ifi regarding the specific region A of the crane 11 and the information Ifj regarding the operating state of the crane 11, displays the operating state of the crane 11 in the specific region A. A plurality of seventh reference coordinates C7 for displaying an oil sound gauge, various warning lights, etc. are calculated.

The image processing device 9 generates information for displaying a plurality of seventh reference coordinates C7 on the display screen of the transmissive display 4 as a seventh virtual image Im7 viewed from the operator's viewpoint.

That is, the image processing device 9 displays information about the crane 11, such as an engine rpm meter, oil pressure gauge, oil sound gauge, various warning lights, etc., which indicate the operating state of the crane in the specific area A viewed from the operator's viewpoint. It is generated as information If7 related to the seventh virtual image Im7.

The image processing device 9 transmits information If7 related to the generated seventh virtual image Im7 of the crane 11 to the display device control device 7 .

The display device control device 7 acquires information If7 regarding the seventh virtual image Im7 of the crane 11 via the communication device 8. The display device control device 7 displays the seventh virtual image Im7 on the specific area A of the transmissive display 4 based on the information If7 on the seventh virtual image Im7 of the crane 11 . Here, the seventh virtual image Im7 is an image relating to an engine rpm meter, oil pressure gauge, oil sound gauge and various warning lights that indicate the operating state of the crane as seen from the operator's viewpoint.

The operator can use the transmissive display device 2 to superimpose an engine speed gauge, oil pressure gauge, oil sound gauge, various warning lights, etc. indicating the operating state of the crane 11 in the actual scenery Ls. recognize.

Therefore, the operator can visually recognize the indicator related to the operation state of the crane 11 without moving his/her line of sight to the control panel in the cabin 27, so that the operator can operate the crane 11 efficiently.

<Trajectory display during combined operation>
As shown in FIG. 17 , the steering support device 1 displays on the transmissive display device 2, as a modified example of the first virtual image Im1, the trajectory when a plurality of operating tools among the crane device operating tools 29 are operated together. may

When the image processing device 9 acquires a plurality of operation signals of the plurality of operation tools among the crane device operation tools 29, the image processing device 9 calculates the current position of the main hook 20a when each part moves at a speed based on the acquired plurality of operation signals. Generate information to display the trajectory from Lp.

According to the generated information, the display device control device 7 displays, as a modified example of the first virtual image Im1, the trajectory of the main hook 20a from the current position Lp when each part moves at a speed based on the plurality of operation signals. do.

The first virtual image Im1, which is the trajectory from the current position Lp of the main hook 20a, can be changed to an arbitrary trajectory by operating a plurality of operating tools among the crane device operating tools 29.

The first virtual image Im1 changes from an arc trajectory passing through the current position Lp to an elliptical trajectory, for example, by operating the turning operation tool 29a and the hoisting operation tool 29c of the boom 19 together. Therefore, when the first virtual image Im1 passes through the target position Lt by operating the crane device operation tool 29, the operator maintains the current operation amount of the crane device operation tool 29, thereby moving the main hook 20a. can be moved to the target position Lt.

<Other embodiments>
In each of the above-described embodiments, the steering support device 1 for the crane 11 has a first virtual image Im1, a second virtual image Im2, a third virtual image Im3, a fourth virtual image Im4, a fifth virtual image Im5, and a sixth virtual image Im5. Either Im6 or the seventh virtual image Im7 is displayed on the transmissive display device 2. FIG.

However, the operation support device 1 of the crane 11 may allow the user of the transmissive display device 2 to arbitrarily select and display at least one of the first virtual image Im1 to the seventh virtual image Im7. Moreover, the operation support device 1 for the crane 11 is not limited to the virtual image displayed in the above-described embodiment, and may display a virtual image that supports the operation of the working machine.

Further, as the embodiment described above, the transmissive display device 2 of the operation support device 1 is attached to the operator who operates the crane 11 within the cabin 27 of the crane 11 . However, the transmissive display device 2 may be attached to an operator who remotely operates the crane 11 from the outside of the crane 11 .

For example, when the operator who remotely operates the crane 11 in the vicinity of the load W to be conveyed is wearing the transmissive display device 2, the operator can see the first virtual image Im1 and the second virtual image Im1 on the transmissive display device 2. By displaying Im2, the current position Lp of the main hook 20a with respect to the position of the load W can be visually recognized more accurately.

Further, in each of the above-described embodiments, the transmission type display device 2 of the operation support device 1 for the crane 11 is attached to the operator of the crane 11 . However, the transmissive display device 2 may be worn by a worker other than the operator of the crane 11 .

For example, when the slinging person is wearing the transmissive display device 2, the slinging person displays the first virtual image Im1 and the third virtual image Im3 on the transmissive display device 2, thereby displaying the main hook on the load W. The operator of the crane 11 can be informed of the current position Lp and the like of the crane 20a more accurately.

Further, in each of the above-described embodiments, the transmission type display device 2 of the operation support device 1 for the crane 11 is attached to the operator of the crane 11 . However, the transmissive display device 2 may be worn by the operator of the crane 11 and at least one of the workers other than the operator of the crane 11 .

For example, when the operator of the crane 11 and the slinger are each wearing the transparent display device 2 , the operation support device 1 of the crane 11 displays the same information about the attitude of the crane 11 on each of the transparent display devices 2 . The same or a different kind of virtual image Im based on Ifc is displayed. As a result, the operator of the crane 11 and the slinger visually recognize the same virtual image Im based on the information Ifc regarding the posture of the crane 11 within the field of view at each position. etc. can be shared more accurately.

Further, as the above-described embodiment, the steering assistance device 1 is configured as the steering assistance device 1 for the crane 11 . However, the steering assistance device may be configured as a steering assistance device for an aerial work vehicle, which is a working machine.

The control support device for the vehicle for work at height provides information on the position of the user, information on the position of the vehicle for work at height, information on the posture of the vehicle for work at height, and a reference plane that serves as a reference for displaying a virtual image via a communication device. Obtain information about the reference plane, including the position and orientation of the . The image processing device of the operation support device for the vehicle for aerial work can arbitrarily show the current position of the bucket, which is a reference member moved by the vehicle for aerial work, and the trajectory from the current position, viewed from the viewpoint of the operator of the vehicle for aerial work. information about the virtual image to be displayed on the display screen of the transmissive display device as the virtual image on the reference plane of the . The transmissive display device acquires information about a virtual image of the aerial work vehicle and displays the virtual image on the display screen.

Further, in each of the above-described embodiments, the operation support device 1 of the crane 11 displays at least one of the first virtual image Im1 to the seventh virtual image Im7 on the transmissive display device 2.

However, the steering support device 1 of the crane 11 may display at least one of the first virtual image Im1 to the seventh virtual image Im7 on the image captured by the camera. At least one of the first virtual image Im1 to the seventh virtual image Im7 can be operated on a display screen such as a touch panel.

Further, in each of the above-described embodiments, the transmissive display device 2 acquires the information Ifa about the user's position necessary for displaying the virtual image Im on the transmissive display 4 by the GNSS receiver 5 for the display device. there is

However, the transmissive display device may calculate the position of the transmissive display device by recognizing a marker positioned at a specific position. The transmissive display device calculates the position of the transmissive display device, for example, by recognizing markers located in the cabin of the crane. In addition, the transmissive display device calculates the position by an inertial measurement unit (IMU) based on the calculated position.

Also, in each of the above-described embodiments, the transmissive display device 2 is a wearable terminal worn by the user on the body. However, the transmissive display device may be any display device that superimposes and visually recognizes a virtual image viewed from the user's viewpoint on a display screen through which the scenery viewed from the user's viewpoint is transmitted. The transmissive display device may be, for example, a head-up display that uses the windshield of the cabin as a display screen to display a virtual image seen from the viewpoint of the operator seated in the operator's seat.

The above-described embodiment merely shows a typical form, and various modifications can be made without departing from the gist of one embodiment. It goes without saying that it can be embodied in various forms, and the scope of the present invention is indicated by the description of the scope of the claims. Including changes.

The disclosure contents of the specification, drawings, and abstract contained in the Japanese application of Japanese Patent Application No. 2021-075393 filed on April 27, 2021 are incorporated herein by reference.

The present invention can be applied to various work machines having booms that can swivel, hoist, and extend and retract.

1 steering support device 2 transmissive display device 3 frame 4 transmissive display 5 GNSS receiver for display device 6 camera for display device 7 control device for display device 8 communication device 9 image processing device 11 crane 20a main hook (reference member)
30 Crane control device Lp Current position Lt Target position S1 Reference surface S2 New reference surface Ifa User position information Ifb Crane position information Ifc Crane posture information Ifd Reference surface Ife Main hook target position Information Iff Information on the new reference plane Ifg Information on the direction and speed of movement of the main hook Ifh Information on the route of the main hook Ifi Information on the specific area of the crane Ifj Information on the operating state of the crane Im1 First virtual image C1 First reference coordinates Information on the first virtual image about the If1 crane

Claims (10)

1. A steering support device for assisting the operation of a work machine having a boom capable of turning, hoisting and telescoping,
   a transparent display,
   a setting unit that sets a reference plane for the scenery seen through the display;
   a control unit that displays on the display a virtual image for assisting operation of the work machine so as to overlap with the reference plane displayed on the display;
   Pilot assistance device.
2. The virtual image is an image showing the current position of a reference member that moves together with the boom, and the reference member moves from the current position with at least one movement of the boom selected from turning, undulating, and telescoping. 2. The steering support device according to claim 1, which is an image showing an actual trajectory.
3. The virtual image is an image showing a target position of a reference member that moves together with the boom, and the reference member moves from the target position in accordance with at least one movement of the boom selected from turning, undulating, and telescoping. 2. The steering support device according to claim 1, which is an image showing an actual trajectory.
4. The steering support device according to claim 1, wherein the virtual image includes an image showing a line parallel to the vertical direction passing through the current position and/or the target position of the reference member that moves with the boom.
5. The steering support device according to claim 1, wherein the virtual image includes a vector image indicating the moving speed and moving direction of the reference member that moves with the boom.
6. The steering assistance device according to claim 1, wherein the control unit controls display on the display based on a hand motion of a user of the steering assistance device captured by a camera.
7. The steering assistance device according to claim 1, wherein the reference plane is the ground surface in the landscape or a plane arbitrarily selected by the user of the steering assistance device.
8. The steering assistance device according to claim 1, further comprising a frame that supports the display and can be worn by a user of the steering assistance device.
9. The steering assistance device according to claim 8, wherein said frame has a pair of temples supported by said user's ears.
10. a boom that can swivel, hoist, and extend and retract;
    A steering support device according to claim 1,
    working machine.

Images