WO2017022042A1 - 作業機械の表示システム、および作業機械の表示方法 - Google Patents
作業機械の表示システム、および作業機械の表示方法 Download PDFInfo
- Publication number
- WO2017022042A1 WO2017022042A1 PCT/JP2015/071886 JP2015071886W WO2017022042A1 WO 2017022042 A1 WO2017022042 A1 WO 2017022042A1 JP 2015071886 W JP2015071886 W JP 2015071886W WO 2017022042 A1 WO2017022042 A1 WO 2017022042A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- control unit
- work machine
- time
- display
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 14
- 238000012545 processing Methods 0.000 claims abstract description 38
- 230000005856 abnormality Effects 0.000 claims description 40
- 239000002131 composite material Substances 0.000 claims description 13
- 230000002093 peripheral effect Effects 0.000 claims description 11
- 238000012544 monitoring process Methods 0.000 abstract description 34
- 230000000694 effects Effects 0.000 abstract description 2
- 240000004050 Pentaglottis sempervirens Species 0.000 description 9
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 9
- 239000000203 mixture Substances 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 239000000446 fuel Substances 0.000 description 7
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 6
- 239000003921 oil Substances 0.000 description 5
- 230000004397 blinking Effects 0.000 description 4
- 239000010720 hydraulic oil Substances 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012806 monitoring device Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 239000002826 coolant Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/04—Diagnosis, testing or measuring for television systems or their details for receivers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
- B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/24—Safety devices, e.g. for preventing overload
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/261—Surveying the work-site to be treated
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
Definitions
- the present invention relates to a work machine display system and a work machine display method capable of quickly grasping an abnormality in a predetermined process of a peripheral monitoring control unit (display control unit) including an image processing unit with a simple configuration.
- multiple cameras are installed behind the vehicle body and on the side of the vehicle body so that the operator can visually recognize obstacles around the vehicle body. There is something that is projected on the monitor in the cab of the work machine.
- an apparatus that generates an overhead view image by converting an image of each camera into an upper viewpoint and then combining the images of each camera so that the surroundings of the vehicle body can be simultaneously monitored and displayed on the monitor.
- Patent Document 1 describes that an icon indicating the arrangement position of a camera that has acquired a selected single camera image is displayed on the image. Thereby, it is possible to easily recognize which camera is a single camera image acquired by.
- image processing such as generating a bird's-eye view image is a work with a heavy load, and therefore processing by a monitor control unit for displaying machine gauge information (such as engine water temperature and fuel remaining amount) on a display unit such as a monitor in general.
- machine gauge information such as engine water temperature and fuel remaining amount
- processing is performed by an image processing controller (periphery monitoring control unit) having a general-purpose image processing accelerator that is separate from the monitor.
- the overhead image generated by the image processing controller is output as a video signal to the display unit of the monitor, and the overhead image is displayed on the display unit.
- the image processing controller sends the same overhead image to the display unit even though the scene captured by the camera is changing, such as when the work machine is moving. There is a possibility to continue. In this case, the operator has detected that a failure has occurred in the image processing controller as long as the overhead image is displayed for the operator, even though the proper images captured by the plurality of cameras cannot be displayed on the display unit. I can't notice.
- the present invention has been made in view of the above, and is a display of a work machine that can quickly grasp an abnormality of a predetermined process of a peripheral monitoring control unit (display control unit) including an image processing unit with a simple configuration. It is an object to provide a system and a display method of a work machine.
- a display system for a work machine generates one or a plurality of cameras that acquire the surrounding state of the work machine, a time-change image, and the 1
- a display control unit that outputs an image obtained by performing predetermined processing on an image captured by one or a plurality of cameras as a video signal, and the time-change image and the predetermined image based on the video signal output from the display control unit
- a monitor that displays the processed image, and an abnormality determination unit that determines that an abnormality has occurred in the display control unit when the time-change image does not change for a predetermined time.
- the display system for a work machine is characterized in that, in the above invention, the display control unit and the abnormality determination unit are provided in separate controllers, respectively.
- the image subjected to the predetermined process by the display control unit is a single camera image and / or an overhead image.
- the work machine display system is characterized in that, in the above-mentioned invention, the abnormality determination unit notifies when an abnormality has occurred in the display control unit.
- the notification includes displaying a message or an icon on the monitor informing the display control unit that an abnormality has occurred.
- the time-change image generated by the display control unit is provided in an outer frame of a single camera image position icon.
- the work machine display system is characterized in that, in the above invention, the time-change image generated by the display control unit is provided in a display area of a single camera image or an overhead image or a peripheral area thereof. To do.
- the time-change image generated by the display control unit is provided in any one of the composite video display regions generated by the display control unit. It is characterized by.
- the abnormality determination unit is characterized by determining abnormality in units of images.
- a display method for a work machine includes one or a plurality of cameras that acquire a surrounding state of the work machine, and a monitor that displays predetermined information.
- a display control step for outputting an image obtained by performing predetermined processing on an image captured by one or a plurality of cameras as a video signal, and the time-change image and the predetermined image based on the video signal output from the display control step.
- the periphery monitoring control unit as the display control unit generates a time-change image and outputs an image obtained by performing predetermined processing on images captured by a plurality of cameras as a video signal, and the display unit The time change image and the image subjected to the predetermined process are displayed based on the video signal output from the periphery monitoring control unit, and the monitor control unit as the abnormality determination unit displays the time change image for a predetermined time. If there is no change, it is determined and notified that an abnormality has occurred in the predetermined processing of the peripheral monitoring control unit, so that the operator can quickly grasp the abnormalities in the predetermined processing of the peripheral monitoring control unit with a simple configuration. it can.
- FIG. 1 is a perspective view showing an overall configuration of a hydraulic excavator on which a working machine periphery monitoring device according to an embodiment of the present invention is mounted.
- FIG. 2 is a diagram showing the internal arrangement of the cab.
- FIG. 3 is a plan view showing the arrangement of the cameras.
- FIG. 4 is a block diagram showing a detailed configuration of the entire control system of the hydraulic excavator and the periphery monitoring control unit.
- FIG. 5 is an explanatory diagram illustrating the overhead image generation processing by the overhead image generation unit.
- FIG. 6 is a diagram illustrating an example of a time change image.
- FIG. 7 is a diagram illustrating a state in which the standard screen is displayed on the display unit of the monitor.
- FIG. 8 is a flowchart showing an abnormality processing procedure of the periphery monitoring control unit by the monitor control unit.
- FIG. 1 is a perspective view showing an overall configuration of a hydraulic excavator on which a work machine periphery monitoring device according to an embodiment of the present invention is mounted.
- the hydraulic excavator 1 is a hydraulic excavator shown as an example of a work machine.
- the hydraulic excavator 1 includes a self-propelled lower traveling body 2, an upper revolving body 3 that is pivotably installed on the lower traveling body 2, and a working machine that bends and undulates at the front center of the upper revolving body 3. 4.
- the work machine 4 includes a boom 4a, an arm 4b, a bucket 4c, a boom cylinder 4d, an arm cylinder 4e, and a bucket cylinder 4f.
- the driver's cab 6 is installed in the front left part of the upper swing body 3.
- a power container 7 is mounted on the rear part of the upper swing body 3.
- an engine, a hydraulic pump, a radiator, an oil cooler, and the like are accommodated.
- a counterweight 8 is attached to the rear end portion of the upper swing body 3.
- FIG. 2 is a diagram showing the internal arrangement of the cab 6.
- the operation levers 11 and 12 are disposed on the left and right of the operator seat 10 in the cab 6, for example.
- the operation lever 11 corresponds to the turning operation of the upper swing body 3 and the operation of the arm 4b
- the operation lever 12 corresponds to the operation of the boom 4a and the operation of the bucket 4c.
- the combination of the operation of the operation levers 11 and 12 and which work machine is operated is not limited to that of the present embodiment.
- the traveling lever 13 and the traveling lever 14 can perform forward and backward movement of the left and right crawler tracks of the lower traveling body 2 according to the operation.
- a monitor 15 is disposed in front of the cab 6.
- the monitor 15 includes a display unit 16 and an operation unit 17.
- the display unit 16 is a display device such as a liquid crystal display, and shows various information including the state of the engine, the hydraulic pump, etc., for example, mechanical information such as engine water temperature, oil temperature, remaining amount of fuel, and warnings indicating device abnormalities. Information, menu screens, maintenance screens, etc. can be displayed.
- the monitor 15 displays an image (single camera image) taken by a camera, which will be described later, or an image (overhead image) generated by combining a plurality of camera images after being converted into an upper viewpoint. Can do.
- the operation unit 17 includes a function switch having a plurality of functions corresponding to the display content of the display unit 16, and a switch for performing various settings such as a work mode, a travel mode, and an air conditioner.
- the monitor 15 may be configured by a touch panel in which the display unit 16 and the operation unit 17 are integrated. Further, the operation unit 17 may be provided separately from the display unit 16 and the operation unit 17 may be provided in another place in the cab.
- the cameras C1 to C4 are attached to the upper swing body 3.
- the camera C ⁇ b> 1 is attached to the rear of the upper swing body 3 and images the rear of the upper swing body 3.
- the camera C ⁇ b> 2 is attached to the right side of the upper swing body 3 and images the right rear of the upper swing body 3.
- the camera C3 is attached to the right side of the upper swing body 3 and images the right front of the upper swing body 3.
- the camera C4 is attached to the left side of the upper swing body 3 and images the left side of the upper swing body 3.
- each of the cameras C1 to C4 has a visual field range of a predetermined angle in the left-right direction and the height direction.
- a CCD (Charge-Coupled Device) camera can be used as the camera.
- Each of the cameras C1 to C4 may have a wide dynamic range function.
- FIG. 4 is a block diagram showing a configuration of the entire control system of the hydraulic excavator 1.
- the periphery monitoring control unit 40 display control unit
- the communication control unit 42 communication control unit
- the monitor control unit 43 abnormality determination unit
- the engine control unit 44 the pump control unit 45, etc.
- the monitor control unit 43 is connected to the monitor 15 and functions as a display control unit for the monitor 15.
- the monitor control unit 43 performs input / output control of various information of the work machine such as information transmitted from various sensors via the CAN 41 and information input via a switch provided on the monitor 15.
- the monitor 15 can perform control to display machine information such as engine water temperature, oil temperature, and remaining amount of fuel, warning information indicating an abnormality of the device, a menu screen, a maintenance screen, and the like.
- the peripheral monitoring control unit 40 is connected to the camera group C composed of a plurality of cameras C1 to C4 and the monitor 15, and, as will be described later, the single camera image and the overhead view image of any of the plurality of cameras. Output the combined image to the monitor.
- the periphery monitoring control unit 40 includes an image processing unit 51 and a display control unit 53. Further, the image processing unit 51 includes an overhead image generation unit 54, a single camera image generation unit 58, a time change image generation unit 55, and an image composition unit 56.
- the overhead image generation unit 54 generates an overhead image 61 based on the images obtained from the cameras C1 to C4. As shown in FIG. 5, the overhead image generation unit 54 converts the images P1 to P4 obtained from the cameras C1 to C4 into an upper viewpoint image. That is, the overhead image generation unit 54 converts the image into an image viewed from a predetermined virtual viewpoint located above the excavator 1.
- the overhead image generation unit 54 performs image conversion to be projected from a virtual viewpoint above the excavator 1 onto a predetermined virtual projection plane corresponding to the ground level. Thereafter, the overhead image generation unit 54 cuts out the converted images P11 to P14 corresponding to the regions E1 to E4 of the frame displaying the overhead image, and synthesizes the converted images P11 to P14 in the frame.
- the bird's-eye view image 61 generated by the bird's-eye view image generation unit 54 is preliminarily pasted with an image P viewed from the top of the hydraulic excavator 1 so that the positional relationship with the hydraulic excavator is clear.
- the information of the bird's-eye view image 61 is input to the image composition unit 56. Note that since the region E0 is visible from the cab 6, no overhead image is generated.
- four cameras C1 to C4 are used. However, any number of cameras may be used, for example, three cameras C1, C2, and C4 may be used. In this case, the camera C2 images the right side in the same manner as the camera C4.
- the single camera image generation unit 58 outputs the single camera image 62 of any one of the images captured by the cameras C1 to C4 to the image composition unit 56. Specifically, when the operation unit 17 switches the single camera image 62 by the operator, the signal is transmitted to the single camera image generation unit 58 via the monitor control unit 43 and the CAN 41, and the single camera image generation unit 58 outputs the single camera image 62 of the corresponding camera that has received the command to the image composition unit 56.
- the time change image generation unit 55 generates a time change image 63 that changes with time.
- the single camera image position icon 70 in the time change image 63 shown in FIG. 6 indicates which direction (rear, right rear, right front) the single camera image 62 displayed on the display unit 16 of the monitor 15 is relative to the excavator.
- the rectangle at the center of the single camera image position icon 70 represents a plan view of the excavator, and the area corresponding to the direction of the single camera image displayed with the upper side of the single camera image position icon 70 as the front side is colored. It is displayed at the beginning.
- FIGS. 6 and 7 show that a single camera image of a rear camera that captures the lower area of the central rectangle, that is, the rear of the excavator is displayed.
- the outer frame 71 of the single camera image position icon 70 blinks with a predetermined color as time passes. For example, the outer frame 71 is displayed in yellow-green when it is turned on, and the outer frame 71 is displayed in black when it is turned off. That is, yellow-green and black are alternately switched over time.
- the periphery monitoring control unit 40 may continue to output the same image to the monitor.
- the outer frame 71 is yellowish green. It does not change from the state displayed with or from the black state. Therefore, the operator can recognize that the periphery monitoring control unit 40 is executing normal image processing by checking whether or not the outer frame 71 portion is blinking.
- the combination of the single camera image position icon 70 and the outer frame 71 is generated by the time change image generation unit 55 as the time change image 63.
- the partial area 72 is an area provided at a corner of the outer frame 71, and, for example, yellow-green and black are alternately switched over time as in the other areas of the outer frame 71.
- time-varying image 63 fluctuates with the passage of time by detecting whether or not the value of the video signal such as luminance and RGB values in the partial region 72 has changed. Judging.
- the time change image 63 generated by the time change image generation unit 55 is input to the image composition unit 56.
- the above-described overhead image 61, the single camera image 62 captured by each of the cameras C1 to C4, and the time change image 63 are input to the image composition unit 56.
- the image synthesis unit 56 generates an image obtained by synthesizing the overhead image 61, the single camera image 62, and the time change image 63 described above based on a control signal instructed from the monitor control unit 43 via the CAN 41. An image obtained by combining these three is output as a video signal S1 to the display unit 16 via the display control unit 53.
- the image composition unit 56 does not receive an output instruction of a composite image of the overhead image 61, the single camera image 62, and the time change image 63 from the monitor control unit 43 (for example, when a menu screen is selected). The image composition unit 56 does not output the composite image to the monitor.
- FIG. 7 is a diagram illustrating a state in which an example of a standard screen is displayed on the display unit 16 of the monitor 15.
- the standard screen is an initial screen that is displayed after a certain time has elapsed since the key switch was turned on.
- Various information on the work machine for example, machine information such as engine water temperature, oil temperature, and remaining amount of fuel, equipment abnormalities, etc. Warning information, a menu screen, a maintenance screen, an overhead image, a single camera image, and the like are displayed.
- an overhead image 61 is displayed on the left of the display unit 16, and a single camera image 62 is displayed on the right.
- a single camera image 62 in FIG. 7 is an image obtained by imaging the rear by the camera C ⁇ b> 1 provided behind the upper swing body 3.
- a guide line D indicating a guide of the distance and direction from the turning center is displayed.
- a time change image 63 is displayed in the upper right of the single camera image 62.
- the bird's-eye view image 61, the single camera image 62, and the time change image 63 are images generated on the periphery monitoring control unit 40 side, and an area in which these three images are displayed is defined as a composite video display area 60.
- the bird's-eye view image 61 is displayed on the left side and the single camera image 62 is displayed on the right side, but the display positions of both may be reversed.
- the composite video display area 60 has an engine water temperature gauge G1 indicating the coolant temperature of the engine, a hydraulic oil temperature gauge G2 indicating the oil temperature of the hydraulic oil in the hydraulic circuit, and a remaining fuel level from the left.
- a fuel level gauge G3 is displayed.
- the hydraulic oil temperature gauge G2 may not be displayed.
- the function switches F1 to F6 are switches for inputting signals corresponding to icons displayed on the display unit 16 immediately above each switch.
- an icon I1 for switching the single camera image 62 to full screen display is displayed above the function switch F3.
- an icon I2 for switching the camera of the single camera image 62 is displayed above the function switch F4.
- An icon I3 for switching to the menu screen is displayed above the function switch F6.
- the menu screen is a screen used when confirming various settings and conditions of the work machine.
- the icons I1 to I3 corresponding to the function switches may not be displayed, and the corresponding switches may be provided in another operation unit or the like.
- the icon I4 displays the service meter value.
- the icon I5 displays the setting state of the work mode.
- FIG. 8 is a flowchart showing an abnormality processing procedure of the periphery monitoring control unit 40 by the monitor control unit 43.
- the monitor control unit 43 functions as an “abnormality determination unit” in the present invention. As shown in FIG. 8, first, the monitor control unit 43 requests whether the monitor control unit 43 requests a composite image of the overhead view image 61, the single camera image 62, and the time change image 63 from the periphery monitoring control unit 40. It is determined whether or not (step S101).
- the monitor control unit 43 When the monitor control unit 43 is requesting a composite image (step S101, Yes), the monitor control unit 43 combines the composite video display area 60 output from the peripheral monitoring control unit 40 to the monitor 15. It is determined whether the time-varying image 63 in the image has not changed for a predetermined time (step S102).
- whether or not the image of the time change image 63 has not changed for a predetermined time is determined by, for example, enlarging the time change image 63 in FIG. 7 and the time change of the video signal in the partial region 72 of the outer frame 71 in FIG.
- the determination may be made from temporal changes such as luminance and RGB values. How long the “predetermined time” is set may be set as appropriate, for example, depending on the blinking speed of the outer frame 71 portion.
- step S102 If the image of the time-varying image 63 has not changed for a predetermined time (step S102, Yes), an error notification that the image processing in the periphery monitoring control unit 40 is abnormal is performed (step S103), and an error is detected.
- the operator is notified in some form of the occurrence.
- error notification for example, a message or icon may be displayed on the display unit 16 of the monitor 15, or may be displayed by a lamp or the like in a place other than the monitor. Also, a buzzer notification by voice may be used.
- step S101 when the monitor control unit 43 does not request a composite image from the periphery monitoring control unit 40 (step S101, No), or when the image of the time change image 63 has changed for a predetermined time (step S102, No). Returning to step S101, the above-described processing is repeated.
- step S101 is that if the monitor control unit 43 does not instruct the peripheral monitoring controller 40 to generate the composite image, that is, a menu screen different from the standard screen in FIG. 7, for example, is displayed.
- the composite image that is, a menu screen different from the standard screen in FIG. 7, for example.
- the time change image 63 is displayed on the standard screen
- the image that changes with the passage of time is not displayed in the area where the time change image 63 is displayed, and the same image continues to be displayed. This is because it is determined that there is an abnormality although no abnormality has occurred in the control unit 40.
- the outer frame 71 of the single camera image position icon 70 is the time-change image 63, and the luminance, RGB values, and other video signal time changes in the partial area 72 of the outer frame 70 are determined. It has been detected whether or not there is an abnormality in the image processing of the periphery monitoring control unit 40.
- the present invention is not limited to this, and any region in the synthesized video display region 60 generated by the periphery monitoring control unit 40 such as a partial region in the overhead image 61 or the single camera image 62 or the periphery of those images may change over time.
- An image may be generated.
- the time-change image to be generated may be a size that can be visually recognized.
- the outer frame 71 of the time change image 63 blinks, it is preferable that the outer frame 71 is visible. This is because the operator can recognize that the peripheral monitoring control unit 40 is executing normal image processing by visually recognizing blinking of the outer frame 71 portion.
- the operator can recognize whether or not there is an abnormality in the image processing of the periphery monitoring control unit 40 without frequently checking whether or not the outer frame 71 portion is blinking. This has the effect of reducing the confirmation load.
- the overhead image 61 and the single camera image 62 are displayed on the standard screen at the same time.
- the present invention is not limited to this, and only the overhead image or the single camera image may be displayed.
- the overhead image 61 and the single camera image 62 may not be displayed on the standard screen, but may be displayed by screen switching.
- an image obtained by performing predetermined processing on an image captured by one or a plurality of cameras may be an overhead image, a single camera image, or an image including both, and a composite video display. An image displayed in the area 60 may be used.
- the periphery monitoring control unit 40 (display control unit) and the monitor control unit 43 (abnormality determination unit) may be provided in one controller (device), or may be provided in different controllers (devices). The structure provided may be sufficient.
- time change image 63 only one time change image 63 is provided.
- a time change image corresponding to each image is prepared, and each image is displayed. You may make it detect whether abnormality has generate
- the said embodiment demonstrated with the structure which provided the monitor 15 in the operator's cab 6 of the working machine, the remote operation room for performing remote operation of other places, for example, a working machine, and the some in the construction site
- a configuration may be adopted in which a display unit is provided in a control room that manages and controls the work machine as a whole.
- the monitor 15 may be configured as a mobile terminal (for example, it may be removable), and an operator may be able to see an overhead image displayed on a display unit in the mobile terminal.
- some communication means is provided for each of the work machine and the operation room / control room / mobile terminal. May be provided to transmit and receive information such as a bird's-eye view image.
Abstract
Description
まず、図1は、本発明の実施の形態である作業機械の周辺監視装置が搭載される油圧ショベルの全体構成を示す斜視図である。この油圧ショベル1は、作業機械の一例として示した油圧ショベルである。油圧ショベル1は、自走可能な下部走行体2と、下部走行体2上に旋回自在に設置された上部旋回体3と、上部旋回体3の前部中央に屈曲起伏自在に動作する作業機4とを備える。
図2は、運転室6の内部配置を示す図である。図2に示すように、操作レバー11,12は、例えば運転室6内のオペレータシート10の左右に配置される。操作レバー11は上部旋回体3の旋回動作とアーム4bの動作に対応しており、操作レバー12はブーム4aの動作とバケット4cの動作に対応している。尚、操作レバー11,12の操作とどの作業機が動作するかの組み合わせは、本実施形態のものには限られない。
カメラC1~C4は、上部旋回体3に取り付けられる。図3に示すように、カメラC1は、上部旋回体3の後方に取り付けられて上部旋回体3の後方を撮像する。また、カメラC2は、上部旋回体3の右側方に取り付けられて上部旋回体3の右後方を撮像する。また、カメラC3は、上部旋回体3の右側方に取り付けられて上部旋回体3の右前方を撮像する。また、カメラC4は、上部旋回体3の左側方に取り付けられて上部旋回体3の左側方を撮像する。なお、例えば、各カメラC1~C4は、左右方向および高さ方向において所定角度の視野範囲を有する。カメラとしてCCD(Charge-Coupled Device)カメラを用いることができる。また、各カメラC1~C4は、ワイドダイナミックレンジ(Wide Dynamic Range)機能を有してもよい。
図4は、油圧ショベル1の制御系全体の構成を示すブロック図である。図4に示すように、周辺監視制御部40(表示制御部)、通信制御部42、モニタ制御部43(異常判断部)、エンジン制御部44、ポンプ制御部45等が、車内ネットワークの一つであるCAN41に接続される。
図7は、モニタ15の表示部16に標準画面の一例が表示された状態を示す図である。標準画面は、キースイッチオン後、一定時間経過した状態で表示される初期画面であり、作業機械の各種情報、例えば、エンジン水温、油温、燃料の残量等の機械情報、機器の異常などを示す警告情報、メニュー画面やメンテナンス画面、俯瞰画像、単カメラ画像などが表示される。
図8は、モニタ制御部43による周辺監視制御部40の異常処理手順を示すフローチャートである。モニタ制御部43が、本発明における「異常判断部」として機能する。図8に示すように、まずモニタ制御部43は、モニタ制御部43が、周辺監視制御部40に対して、俯瞰画像61、単カメラ画像62及び時間変化画像63の合成画像を要求しているか否かを判断する(ステップS101)。
2 下部走行体
3 上部旋回体
4 作業機
4a ブーム
4b アーム
4c バケット
4d ブームシリンダ
4e アームシリンダ
4f バケットシリンダ
6 運転室
7 パワーコンテナ
8 カウンタウェイト
10 オペレータシート
11,12 操作レバー
13,14 走行レバー
15 モニタ
16 表示部
17 操作部
40 周辺監視制御部(表示制御部)
42 通信制御部
43 モニタ制御部(異常判断部)
44 エンジン制御部
45 ポンプ制御部
51 画像処理部
53 表示制御部
54 俯瞰画像生成部
55 時間変化画像生成部
56 画像合成部
58 単カメラ画像生成部
60 合成映像表示領域
61 俯瞰画像
62 単カメラ画像
63 時間変化画像
70 単カメラ画像位置アイコン
71 外枠
72 一部領域
C カメラ群
C1~C4 カメラ
D 目安線
G1 エンジン水温ゲージ
G2 作動油温ゲージ
G3 燃料レベルゲージ
I1~I5 アイコン
S1 映像信号
Claims (10)
- 作業機械の周囲状況を取得する1つもしくは複数のカメラと、
時間変化画像を生成するとともに前記1つもしくは複数のカメラが撮像した画像に対して所定処理を行った画像を映像信号として出力する表示制御部と、
前記表示制御部から出力された映像信号をもとに前記時間変化画像と前記所定処理を行った画像とを表示するモニタと、
前記時間変化画像が所定時間変化しない場合、前記表示制御部に異常が発生したと判断する異常判断部と、
を備えたことを特徴とする作業機械の表示システム。 - 前記表示制御部と前記異常判断部とは、別のコントローラにそれぞれ設けられることを特徴とする請求項1に記載の作業機械の表示システム。
- 前記表示制御部が所定処理を行った画像とは、単カメラ画像及び/または俯瞰画像であることを特徴とする請求項1または2に記載された作業機械の表示システム。
- 前記異常判断部は、前記表示制御部に異常が発生したと判断した場合、報知することを特徴とする請求項1~3のいずれか一つに記載された作業機械の表示システム。
- 前記報知は、前記表示制御部に異常が発生した旨を知らせるメッセージ、又はアイコンを前記モニタに表示することであることを特徴とする請求項4に記載された作業機械の表示システム。
- 前記表示制御部が生成する時間変化画像は、単カメラ画像位置アイコン外枠に設けられることを特徴とする請求項1~5のいずれか一つに記載された作業機械の表示システム。
- 前記表示制御部が生成する時間変化画像は、単カメラ画像または俯瞰画像の表示領域またはその周辺領域に設けられることを特徴とする請求項1~5のいずれか一つに記載された作業機械の表示システム。
- 前記表示制御部が生成する時間変化画像は、前記表示制御部が生成する合成映像表示領域のいずれかの領域に設けられることを特徴とする請求項1~5のいずれか一つに記載された作業機械の表示システム。
- 前記表示制御部が所定処理を行った画像が複数ある場合、前記時間変化画像は、前記複数の画像それぞれに設けられ、前記異常判断部は画像単位で異常を判断することを特徴とする請求項1~8のいずれか一つに記載の作業機械の表示システム。
- 作業機械の周囲状況を取得する1つもしくは複数のカメラと、
所定の情報を表示するモニタと、を備え、
時間変化画像を生成するとともに前記1つもしくは複数のカメラが撮像した画像に対して所定処理を行った画像を映像信号として出力する表示制御ステップと、
前記表示制御ステップから出力された映像信号をもとに前記時間変化画像と前記所定処理を行った画像とを表示するステップと、
前記時間変化画像が所定時間変化しない場合、異常が発生したと判断する異常判断ステップと、
を備えたことを特徴とする作業機械の表示方法。
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201580078750.3A CN107431783A (zh) | 2015-07-31 | 2015-07-31 | 作业机械的显示系统及作业机械的显示方法 |
US15/561,203 US20180295357A1 (en) | 2015-07-31 | 2015-07-31 | Display system of working machine and display method of working machine |
JP2017532270A JP6577034B2 (ja) | 2015-07-31 | 2015-07-31 | 作業機械の表示システム、および作業機械の表示方法 |
KR1020177026226A KR20170118831A (ko) | 2015-07-31 | 2015-07-31 | 작업 기계의 표시 시스템, 및 작업 기계의 표시 방법 |
PCT/JP2015/071886 WO2017022042A1 (ja) | 2015-07-31 | 2015-07-31 | 作業機械の表示システム、および作業機械の表示方法 |
KR1020197018050A KR102027395B1 (ko) | 2015-07-31 | 2015-07-31 | 작업 기계의 표시 시스템, 및 작업 기계의 표시 방법 |
DE112015006345.5T DE112015006345T5 (de) | 2015-07-31 | 2015-07-31 | Anzeigesystem einer arbeitsmaschine und anzeigeverfahren einer arbeitsmaschine |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2015/071886 WO2017022042A1 (ja) | 2015-07-31 | 2015-07-31 | 作業機械の表示システム、および作業機械の表示方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017022042A1 true WO2017022042A1 (ja) | 2017-02-09 |
Family
ID=57943902
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/071886 WO2017022042A1 (ja) | 2015-07-31 | 2015-07-31 | 作業機械の表示システム、および作業機械の表示方法 |
Country Status (6)
Country | Link |
---|---|
US (1) | US20180295357A1 (ja) |
JP (1) | JP6577034B2 (ja) |
KR (2) | KR20170118831A (ja) |
CN (1) | CN107431783A (ja) |
DE (1) | DE112015006345T5 (ja) |
WO (1) | WO2017022042A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020063569A (ja) * | 2018-10-15 | 2020-04-23 | 日立建機株式会社 | 油圧ショベル |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3537403A4 (en) * | 2016-11-01 | 2020-01-08 | Sumitomo (S.H.I.) Construction Machinery Co., Ltd. | SECURITY MANAGEMENT SYSTEM FOR WORKING MACHINE, MANAGEMENT DEVICE, SECURITY MANAGEMENT METHOD |
KR20180095986A (ko) * | 2017-02-20 | 2018-08-29 | 두산인프라코어 주식회사 | 건설 장비의 화면 표시 시스템 및 화면 표시 방법 |
US11131082B2 (en) * | 2019-06-13 | 2021-09-28 | Deere & Company | Work vehicle with a payload tracking system |
WO2021085608A1 (ja) * | 2019-10-31 | 2021-05-06 | 住友建機株式会社 | ショベル管理システム、ショベル用携帯端末、およびショベル用携帯端末に用いられるプログラム |
JP2021143541A (ja) * | 2020-03-13 | 2021-09-24 | コベルコ建機株式会社 | 作業支援サーバ、作業支援方法 |
JP7458850B2 (ja) * | 2020-03-26 | 2024-04-01 | 住友重機械建機クレーン株式会社 | 作業機械の周囲表示装置 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004214970A (ja) * | 2002-12-27 | 2004-07-29 | Sumitomo Electric Ind Ltd | 画像表示システム、画像表示方法及び画像処理装置 |
JP2008283431A (ja) * | 2007-05-10 | 2008-11-20 | Matsushita Electric Ind Co Ltd | 画像処理装置 |
JP2014064192A (ja) * | 2012-09-21 | 2014-04-10 | Komatsu Ltd | 作業車両用周辺監視システム及び作業車両 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5269026B2 (ja) * | 2010-09-29 | 2013-08-21 | 日立建機株式会社 | 作業機械の周囲監視装置 |
JP5341044B2 (ja) * | 2010-09-29 | 2013-11-13 | 日立建機株式会社 | 作業機械の表示システム |
JP5587499B2 (ja) * | 2011-06-07 | 2014-09-10 | 株式会社小松製作所 | 作業車両の周辺監視装置 |
AU2013208525A1 (en) | 2012-01-12 | 2014-08-14 | Hitachi Construction Machinery Co., Ltd. | Periphery monitoring device for self-propelled industrial machine |
JP5411976B1 (ja) * | 2012-09-21 | 2014-02-12 | 株式会社小松製作所 | 作業車両用周辺監視システム及び作業車両 |
-
2015
- 2015-07-31 WO PCT/JP2015/071886 patent/WO2017022042A1/ja active Application Filing
- 2015-07-31 DE DE112015006345.5T patent/DE112015006345T5/de not_active Withdrawn
- 2015-07-31 KR KR1020177026226A patent/KR20170118831A/ko active Application Filing
- 2015-07-31 US US15/561,203 patent/US20180295357A1/en not_active Abandoned
- 2015-07-31 KR KR1020197018050A patent/KR102027395B1/ko active IP Right Grant
- 2015-07-31 JP JP2017532270A patent/JP6577034B2/ja active Active
- 2015-07-31 CN CN201580078750.3A patent/CN107431783A/zh active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004214970A (ja) * | 2002-12-27 | 2004-07-29 | Sumitomo Electric Ind Ltd | 画像表示システム、画像表示方法及び画像処理装置 |
JP2008283431A (ja) * | 2007-05-10 | 2008-11-20 | Matsushita Electric Ind Co Ltd | 画像処理装置 |
JP2014064192A (ja) * | 2012-09-21 | 2014-04-10 | Komatsu Ltd | 作業車両用周辺監視システム及び作業車両 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020063569A (ja) * | 2018-10-15 | 2020-04-23 | 日立建機株式会社 | 油圧ショベル |
WO2020080294A1 (ja) * | 2018-10-15 | 2020-04-23 | 日立建機株式会社 | 油圧ショベル |
Also Published As
Publication number | Publication date |
---|---|
US20180295357A1 (en) | 2018-10-11 |
JP6577034B2 (ja) | 2019-09-18 |
JPWO2017022042A1 (ja) | 2018-03-08 |
KR102027395B1 (ko) | 2019-10-01 |
DE112015006345T5 (de) | 2017-12-07 |
KR20190073621A (ko) | 2019-06-26 |
CN107431783A (zh) | 2017-12-01 |
KR20170118831A (ko) | 2017-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6577034B2 (ja) | 作業機械の表示システム、および作業機械の表示方法 | |
US10337174B2 (en) | Working machine display system, working machine display device, and working machine display for prompt screen transition even when abnormal processing occurs | |
JP7108750B2 (ja) | システムおよび方法 | |
JP6427597B2 (ja) | 作業機械の周辺監視装置及び作業機械の周辺監視方法 | |
JP6216097B2 (ja) | 作業機械の周辺監視装置 | |
JP6200121B2 (ja) | 作業機械の周辺監視装置 | |
EP2717570A1 (en) | Device for monitoring area around working machine | |
JP6071786B2 (ja) | 作業機械の周囲監視装置 | |
JPWO2012157379A1 (ja) | 作業機械の周囲監視装置 | |
JP2013253402A (ja) | 作業機械の周囲監視装置 | |
JP2014161087A (ja) | 作業車両用周辺監視システム及び作業車両 | |
JP6980391B2 (ja) | 作業機械用周辺監視システム | |
JP2016169595A (ja) | 作業機械の周囲監視装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15900355 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2017532270 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20177026226 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15561203 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 112015006345 Country of ref document: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15900355 Country of ref document: EP Kind code of ref document: A1 |