WO2020065738A1 - 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 - Google Patents
作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 Download PDFInfo
- Publication number
- WO2020065738A1 WO2020065738A1 PCT/JP2018/035525 JP2018035525W WO2020065738A1 WO 2020065738 A1 WO2020065738 A1 WO 2020065738A1 JP 2018035525 W JP2018035525 W JP 2018035525W WO 2020065738 A1 WO2020065738 A1 WO 2020065738A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- work machine
- coordinate system
- machine
- work
- image
- Prior art date
Links
- 238000005259 measurement Methods 0.000 title claims abstract description 89
- 238000003384 imaging method Methods 0.000 claims description 28
- 239000003550 marker Substances 0.000 claims description 14
- 239000000284 extract Substances 0.000 claims description 6
- 230000036544 posture Effects 0.000 claims 12
- 238000006243 chemical reaction Methods 0.000 description 21
- 238000000034 method Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 16
- 238000012545 processing Methods 0.000 description 11
- 238000001514 detection method Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 239000000470 constituent Substances 0.000 description 8
- 230000010365 information processing Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 239000003921 oil Substances 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000010720 hydraulic oil Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F3/00—Dredgers; Soil-shifting machines
- E02F3/04—Dredgers; Soil-shifting machines mechanically-driven
- E02F3/28—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets
- E02F3/30—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets with a dipper-arm pivoted on a cantilever beam, i.e. boom
- E02F3/32—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets with a dipper-arm pivoted on a cantilever beam, i.e. boom working downwardly and towards the machine, e.g. with backhoes
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/261—Surveying the work-site to be treated
- E02F9/262—Surveying the work-site to be treated with follow-up actions to control the work tool, e.g. controller
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/264—Sensors and their calibration for indicating the position of the work tool
- E02F9/265—Sensors and their calibration for indicating the position of the work tool with follow-up actions (e.g. control signals sent to actuate the work tool)
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/18—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
- G05B19/19—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/64—Analysis of geometric attributes of convexity or concavity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/39—Robotics, robotics to robotics hand
- G05B2219/39393—Camera detects projected image, compare with reference image, position end effector
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Definitions
- the present invention relates to a work machine outer shape measurement system including a measurement controller for measuring the shape of a work machine mounted on the work machine.
- work machines including hydraulic excavators have (1) detected the position and orientation of work machines (front work machines) such as booms, arms, and buckets, and their sensor information.
- work machines front work machines
- MG Machine guidance
- Some machines have a machine control (MC) function for controlling a work machine according to the above conditions.
- MC machine control
- a hydraulic shovel equipped with such a function displays an image of a side view of the bucket together with the target surface on a monitor in the cab in order to inform an operator of the relative positional relationship between the actual bucket and the target surface. Is done.
- Patent Document 1 discloses the following technology from the viewpoint of reducing operator discomfort when displaying images of a plurality of types of buckets on a monitor. That is, the cited document 1 is a display system of a work machine having a work machine with a bucket attached thereto, and is drawing information for drawing an image of the bucket as viewed from the side using information on the shape and dimensions of the bucket. And a display for displaying an image of the bucket viewed from the side and an image showing a cross section of the terrain based on the drawing information generated by the generator, and the shape and size of the bucket.
- the information of (1) includes, in a side view of the bucket, a distance between a cutting edge of the bucket and a bucket pin for attaching the bucket to the work machine, a straight line connecting the cutting edge and the bucket pin, and a straight line indicating the bottom surface of the bucket.
- the angle, the position of the cutting edge, the position of the bucket pin, and the distance from the portion connecting the bucket to the work implement to the cutting edge that comprises at least one location outside of the bucket, discloses a display system of a work machine.
- a work machine including a bucket is manufactured by a manual operation such as welding by an operator, so that a deformation or a positional shift occurs in the process, and the work machine is finished in a shape different from the design data. Therefore, the outline information of the working machine is generally obtained by performing measurement with a measure or measurement using a total station for each working machine. In these methods, it is necessary to limit the posture of the working machine and to use a large measuring device in order to obtain sufficient measurement accuracy. In addition, in order to accurately display the shape of the working machine on the monitor in accordance with the actual shape, it is necessary to measure as many points as possible in advance, which is a very laborious operation.
- An object of the present invention is to provide a measurement system capable of easily measuring the outer shape information of a work machine, a display system and a control system that assist an operator in accurately forming a target surface by using the measurement system, and a display system and a control system therefor. To provide a working machine equipped with the same.
- the present application includes a plurality of means for solving the above-mentioned problems.
- the work machine outer shape measurement system including a measurement controller for measuring the shape of the work machine mounted on the work machine, A photographing device for photographing the side surface of the work machine, wherein the measurement controller sets a three-dimensional image set in the photographing device based on an image of the work device photographed by the photographing device and internal parameters of the photographing device.
- a position of a plane representing a side surface of the work machine is calculated in a photographing device coordinate system which is a coordinate system, and based on position information in the image of an arbitrary pixel constituting the work machine and the position of the plane on the image.
- the outer shape information of the working machine can be easily measured.
- FIG. 1 is a configuration diagram of a hydraulic shovel and a photographing device according to a first embodiment of the present invention.
- FIG. 1 is a configuration diagram of a system according to a first embodiment of the present invention. The figure showing the coordinate system in a hydraulic shovel.
- FIG. 1 is a functional block diagram of an external shape measurement system for a working machine according to a first embodiment of the present invention. The figure which shows the example of the known point marker attached on the working machine side surface. The figure which shows the positional relationship of the imaging device in the imaging device coordinate system, and the known point marker on the side of a working machine.
- FIG. 4 is a diagram illustrating a positional relationship between images in an image sensor coordinate system. The figure which shows the relationship between an imaging device coordinate system and a working machine coordinate system.
- FIG. 1 is a functional block diagram of a hydraulic shovel system according to a first embodiment of the present invention.
- FIG. 3 is a diagram showing a target plane in a vehicle body coordinate system. The figure which shows the example of the relationship between a working machine and a target surface.
- FIG. 4 is a view showing an example of a screen displayed on a display monitor 18.
- the block diagram of the hydraulic shovel which concerns on 2nd Embodiment of this invention.
- FIG. 7 is a functional block diagram of a system according to a second embodiment of the present invention.
- a hydraulic shovel having a bucket 4 as an attachment at the tip of a working machine (a front working machine) is illustrated.
- the present invention is applied to a hydraulic shovel having an attachment other than the bucket. It does not matter.
- the present invention is applicable to work machines other than hydraulic excavators as long as the work machine has a work machine such as a wheel loader.
- the shape of the measurement controller 20 (at any point on the work machine 1A).
- One or more front members for which measurement of the work machine coordinate system Co3 (position in the later-described) is desired may be referred to as a work machine.
- a photographing device for example, a camera
- working machine for photographing a working machine 1A mounted on a hydraulic excavator (working machine) 1 and an image of the side of the working machine 1A photographed by the photographing device 19 (hereinafter, “working machine”) (May be referred to as “machine side image”)
- a measurement controller 20 that measures information about the shape of the work machine 1A
- a shape controller that is mounted on the excavator 1 and that is calculated by the measurement controller 20.
- a system including a work machine controller 50 for inputting information and executed by the excavator 1 for use in, for example, MG or MC will be described.
- FIG. 1 is a configuration diagram of the excavator 1, the photographing device 19, and the measurement controller 20 according to the embodiment of the present invention.
- FIG. 2 is a configuration diagram of the system of the present embodiment. As shown in FIG. 2, the system according to the present embodiment includes a hydraulic shovel 1 on which a work machine controller 50 is mounted, an imaging device 19 and a measurement controller 20 installed at a position away from the hydraulic shovel 1.
- the photographing device 19 is a camera that photographs a photograph (image) of the side surface of the work machine 1A.
- the measurement controller 20 calculates the position of the plane representing the side surface of the work machine 1A, and based on the position of the plane and the image captured by the imaging device 19, the work machine coordinate system of an arbitrary point on the side surface of the work machine 1A.
- the coordinate value in Co3 and the drawing image of the work machine 1A are generated.
- the work machine controller 50 mounted on the hydraulic excavator 1 provides a machine guidance (MG) function and a machine control (MC) function.
- the work machine controller 50 includes a measurement controller as shape information and drawing information of the work machine 1A for the MG / MC.
- the coordinate values in the work machine coordinate system Co3 of an arbitrary point on the side surface of the work machine 1A output by 20 and the drawn image of the work machine 1A are used.
- the measurement controller 20 and the work machine controller 50 are control devices each having a processing device (for example, a CPU) and a storage device (for example, a semiconductor memory such as a ROM or a RAM) in which a program executed by the processing device is stored.
- the controllers 20 and 50 of the present embodiment include an external device (for example, the photographing device 19, a target plane data input device 37 (see FIG. 9), various sensors 12, 13, 14, 16, 17 and operation levers 10, 11). ), And performs various calculations necessary for generating the coordinate values of the work implement 1A and the drawn image, and displays the information on a display monitor (display device) 18 installed in the cab of the excavator 1.
- Various calculations relating to the operation of the hydraulic excavator 1 are performed. The specific contents of the calculations executed by the measurement controller 20 and the work machine controller 50 will be described later with reference to the functional block diagrams of FIGS.
- a hydraulic shovel 1 is a multi-joint type working machine (a front working machine) configured by connecting a plurality of front members (a boom 2, an arm 3, and a bucket 4) which respectively rotate in a vertical direction. 1) and a vehicle body 1B composed of an upper revolving unit 1BA and a lower traveling unit 1BB, and the base end of the boom 2 located on the base end side of the work implement 1A is vertically turned to the front of the upper revolving unit 1BA. It is movably supported.
- the upper revolving unit 1BA is rotatably mounted on the upper part of the lower traveling unit 1BB.
- a measurement controller 20 On the side of the work machine 1A, there are provided internal parameters (for example, focal length (f), image sensor size (h, w)), number of pixels (h, H) for taking a photograph of the side of the work machine 1A. W), a unit cell size, image center coordinates, etc.) are clearly provided, and a measurement controller 20 is provided.
- focal length f
- image sensor size h, w
- number of pixels for taking a photograph of the side of the work machine 1A.
- W a unit cell size, image center coordinates, etc.
- the imaging device 19 is a monocular camera provided with an imaging element (image sensor) such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).
- image sensor image sensor
- the photographing device 19 outputs photographed image data to the measurement controller 20.
- the photographing device 19 measures depth of arrival (distance information to the subject) using parallax, such as a stereo camera, or emits laser light or the like and measures the arrival time of the reflected light.
- parallax such as a stereo camera, or emits laser light or the like and measures the arrival time of the reflected light.
- a camera that can acquire depth information may be used instead.
- the measurement controller 20 may be built in the photographing device 19.
- the boom 2, the arm 3, the bucket 4, the upper swing body 1BA, and the lower traveling body 1BB are respectively composed of a boom cylinder 5, an arm cylinder 6, a bucket cylinder 7, a swing hydraulic motor 8, and left and right traveling hydraulic motors 9a, 9b (hydraulic actuators). )
- a driven member To form a driven member.
- the operation of the plurality of driven members is performed by a traveling right lever 10a, a traveling left lever 10b, an operating right lever 11a, and an operating left lever 11b installed in the cab on the upper swing body 1BA (these are referred to as the operating levers 10, 11). Is controlled by a pilot pressure generated by being operated by an operator.
- the pilot pressures for driving the plurality of driven members include not only those output by operating the operation levers 10 and 11, but also the plurality of proportional solenoid valves 39 (see FIG. 9) mounted on the excavator 1.
- a part (pressure increasing valve) operates and outputs independently of the operation of the operating levers 10 and 11, or a part (pressure reducing valve) of a plurality of proportional solenoid valves 39 operates and operates the operating levers 10 and 11.
- the pilot pressure outputted by the above is reduced.
- the pilot pressure output from the plurality of proportional solenoid valves 39 (the pressure increasing valve and the pressure reducing valve) activates the MC that operates the boom cylinder 5, the arm cylinder 6, and the bucket cylinder 7 in accordance with predetermined conditions.
- the work machine 1A is provided with a boom angle sensor 12 on the boom pin and an arm angle sensor 13 on the arm pin so that the rotation angles ⁇ , ⁇ , and ⁇ (see FIG. 3) of the boom 2, the arm 3, and the bucket 4 can be measured.
- the bucket angle sensor 14 is attached to the bucket link 15.
- the upper revolving unit 1BA includes a vehicle body front-rear tilt angle sensor 16a that detects a front-rear direction tilt angle ⁇ (see FIG. 3) of the upper revolving unit 1BA (the vehicle body 1B) with respect to a reference plane (for example, a horizontal plane).
- a vehicle body left / right tilt angle sensor 16b for detecting a left / right tilt angle ⁇ (not shown) of the vehicle body 1B) is attached.
- the X-axis and the Z-axis shown in FIG. 3 are originated from a point on the axis of the boom pin (for example, the center point), the Z-axis is the upward direction of the vehicle, the X-axis is the forward direction of the vehicle, and the rightward direction of the vehicle.
- first GNSS antenna 17a and a second GNSS antenna 17b are arranged on the upper swing body 1BA.
- the first GNSS antenna 17a and the second GNSS antenna 17b are antennas for RTK-GNSS (Real Time Kinetics-Global Navigation Satellite Systems) and receive radio waves (navigation signals) transmitted from a plurality of GNSS satellites.
- the work machine controller 50 determines the latitude, longitude and height (ellipsoid height) of each antenna position based on the time required for radio waves transmitted from a plurality of GNSS satellites to reach the first and second GNSS antennas 17a and 17b. ) Can be measured.
- the position and the direction of the excavator 1 (the upper swing body 1BA) in the geographic coordinate system (world coordinate system) Co5 which is a three-dimensional coordinate system can be calculated.
- a configuration may be adopted in which the positions and heights of the first and second GNSS antennas 17a and 17b are calculated by a dedicated receiver, and the calculation results are output to the work machine controller 50.
- the posture information of the work implement 1A calculated from the outputs of the various posture sensors 12, 13, 14, 16 and the calculation based on the reception signals of the GNSS antennas 17a, 17b are calculated.
- An image of the work implement 1A as viewed from the side and a cross-sectional shape of the target plane are displayed based on the position information and the like of the upper swing body 1BA thus obtained.
- An image of the work machine 1A as viewed from the side is generated by the measurement controller 20 based on the work machine side image taken by the photographing device 19.
- the measurement controller 20 generates a coordinate value and a drawn image of the work machine 1A on the work machine coordinate system Co3 based on the work machine side image of the imaging device 19 will be described with reference to the drawings.
- FIG. 4 is a functional block diagram of the measurement controller 20 according to the embodiment of the present invention. As shown in this figure, the measurement controller 20 calculates the coordinate value of the work implement 1A in the work implement coordinate system Co3, and draws the work implement 1A in the work implement coordinate system Co3. A work machine coordinate system drawing image generation unit 22 for generating an image is provided, and receives a side photograph of the work machine 1 ⁇ / b> A photographed by the photographing device 19.
- the work machine coordinate system coordinate calculator 21 calculates a photographing position calculator 23 that calculates the position of a plane representing the side surface of the work machine 1A in the photographing apparatus coordinate system Co1, which is a three-dimensional coordinate system set in the photographing apparatus 19;
- the photographing device coordinate system coordinate conversion unit 24 calculates the coordinate value of the photographing device coordinate system Co1 in the photographing device coordinate system Co1.
- a work machine coordinate system coordinate conversion unit 25 that converts the coordinate value into a coordinate value in the machine coordinate system Co3.
- the work machine coordinate system coordinate calculation unit 21 receives a side image (work machine side image) of the work machine 1A photographed by the photographing device 19 as an input, and sets a work machine of a work machine constituent pixel designated on the work machine side image.
- the coordinate value of the corresponding point in the working machine coordinate system Co3 is output.
- the measurement controller 20 may execute a process of performing distortion correction on the work machine side image based on the internal parameters of the photographing device 19.
- the coordinates of the working machine corresponding points of all pixels on the contour line, which is the boundary line between the working machine side surface and the other part (background), in the working machine coordinate system Co3 ( The case of outputting the working machine coordinate system coordinates) will be described.
- the working machine corresponding points of all the pixels on the working machine side surface on the working machine side image that is, all the pixels within the outline of the working machine side surface
- a method may be adopted.
- the photographing position calculating unit 23 uses a three-dimensional coordinate system set in the photographing device 19 based on an image (side image of the working machine) obtained by photographing the side surface of the work machine 1A by the photographing device 19 and internal parameters of the photographing device 19.
- the position of a plane S1 (see FIG. 8 described later) representing the side surface of the work machine 1A in a certain imaging device coordinate system Co1 is calculated.
- the position of the plane S1 in the imaging device coordinate system Co1 is specified by the equation of the plane S1 in the imaging device coordinate system Co1.
- the photographing position calculation unit 23 when calculating the equation of the plane S1 representing the work machine 1A by the photographing position calculation unit 23, markers whose distances are known on the side surface of the work machine 1A photographed by the photographing device 19 (known point markers). As shown in FIG. 5, a method is adopted in which three 40s are installed at positions forming a triangle. In the present embodiment, the positions (coordinates) of the three known point markers 40 in the photographing device coordinate system Co1 are calculated from the pixel positions of the three markers 40 in the working machine side image, and the plane S1 is calculated from the three points. The equation is being calculated.
- the plane S1 is the operation plane of the work machine 1A (for example, It is preferable to arrange three markers on a plane parallel to the operation plane so as to be parallel to the plane perpendicular to the operation plane, but to place three markers on a plane intersecting the operation plane (that is, a plane not parallel to the operation plane). You may place two markers. Since it is sufficient that a plane can be defined by the known point markers 40, four or more known point markers 40 may be attached to the side surface of the work machine 1A to obtain the equation of the plane S1 where all the markers 40 are located.
- the marker 40 is an object having characteristics such as a predetermined size, color, pattern, shape, and property.
- a marker that reflects light of a specific wavelength a marker that reflects light in a specific direction
- an AR Augmented Reality
- An AR marker used in the technology or a marker including a two-dimensional code such as a QR code (registered trademark) may be used.
- FIG. 5 shows a specific example of the known point marker 40 in the present embodiment.
- three known point markers 40 are installed on the side surface of the bucket 4 so as not to be located on the same straight line, and the plane is calculated by obtaining the coordinate values of the three known point markers 40.
- three straight lines may be drawn on the side surface of the work machine, and the three markers 40 may be arranged at three intersections where the three straight lines intersect.
- FIG. 6 is a diagram showing a positional relationship between the photographing device of the photographing device 19 and the known point markers 40 (P1, P2, P3) on the side surface of the work machine in the photographing device coordinate system Co1.
- the photographing device coordinate system Co1 is a coordinate system in which the origin O is at the lens center of the photographing device 19, the Z axis is the direction of the optical axis, the Y axis is above the photographing device 19, and the X axis is to the right.
- the unit of the coordinate value in the imaging device coordinate system Co1 is a unit of length, such as millimeters.
- Points P1 to P3 are the positions of the known point markers 40, and any point on the side of the working machine similar to the points P1 to P3 is Q.
- Lij is a known value.
- FIG. 7 shows images of the points P1 to P3 and the point Q reflected on the image sensor 35 of the photographing device 19, and points P1 'to P3' and Q 'are the positions of the images of the points P1 to P3 and Q, respectively.
- a two-dimensional coordinate system having the optical axis center at the origin O ', the right direction of the sensor as the U axis, and the upward direction as the V axis is defined as an image sensor coordinate system Co2.
- the photographing device coordinate system coordinate conversion unit 24 determines the arbitrary work mechanism based on the position information of the arbitrary work machine constituent pixel on the work machine side image and the equation calculated by the photographing position calculation unit 23.
- the coordinate value of the work pixel corresponding point of the formed pixel in the imaging device coordinate system Co1 is calculated.
- the photographing device coordinate system coordinate conversion unit 24 extracts the outline of the working machine 1A from the side view image of the working machine taken by the photographing device 19 by image processing, and an arbitrary position located on the extracted outline.
- the coordinate value of the working machine corresponding point in the imaging device coordinate system Co1 is obtained based on the pixel position information and the equation relating to the point Q (the equation of the plane S1).
- the method for obtaining the coordinates in is as follows.
- a point (working machine constituent pixel) at which an arbitrary point Q (working machine corresponding point) on the plane S1 on the side of the working machine is formed on the image sensor is defined as a point Q '.
- the point Q' can be expressed by the following equation (7) in the same manner as in the above equation (1).
- the straight line OQ can be expressed as in the following equation (8) using the above equation (7).
- the position (coordinate) of the point Q (working machine corresponding point) in the photographing device coordinate system Co1 is determined by the equation of the plane S1 representing the side surface of the working machine 1A (the above equation (6)) and the point Q ′ on the image sensor. And the equation of a straight line passing through the origin O (formula (8)).
- the work machine coordinate system coordinate conversion unit 25 converts the coordinate value (the position of the point Q) of the work machine corresponding point in the photographing apparatus coordinate system Co1 into a coordinate value in the work machine coordinate system Co3, and the converted coordinate value (hereinafter, referred to as the coordinate value). (Which may be referred to as “work implement coordinate system coordinate value”) to the work machine controller 50 and the work implement coordinate system drawing image generation unit 22.
- the work machine coordinate system Co3 is a two-dimensional coordinate system defined on a plane S1 representing a side surface of the work machine 1A whose equation has been obtained by the photographing position calculation unit 23, and is a coordinate value in the work machine coordinate system Co3. Is a unit of length, such as millimeters.
- the work machine coordinate system Co3 has the origin as the center of rotation of the work machine 1A, the x-axis in the work machine tip direction, and the y-axis in a direction orthogonal to the x-axis.
- FIG. 8 shows a relationship diagram of the plane S1 representing the side surface of the work machine 1A, the work machine coordinate system Co3, and the photographing device coordinate system Co1 in the present embodiment.
- the bucket 4 is targeted among the plurality of front members 2, 3, 4 constituting the multi-joint type working machine 1 ⁇ / b> A, and the rotation center of the bucket 4 is set as the origin of the working machine coordinate system Co 3.
- the y-axis of the working machine coordinate system Co3 is set in a direction orthogonal to the x-axis.
- the center of rotation on the base end side is used as the origin, and the center of rotation of the arm 3 or the bucket 4 (the front member of each front member) is set.
- the x-axis may be set toward the tip.
- the work machine coordinate system coordinate calculation unit 21 calculates work machine coordinate system coordinates for work machine corresponding points of all pixels on the contour line of the work machine 1A, and the work machine coordinate system drawing image generation unit 22 Is described as an example, but the coordinates of the working machine coordinate system Co3 may be calculated and output for the working machine corresponding points of some pixels on the contour line.
- the pixels on the contour and one or more pixels included in the contour, the pixels on the contour and all the pixels included in the contour, and all the working mechanisms by the user using some input interface are calculated and output.
- the accuracy of the control of the work machine 1A in the MC and the accuracy of the distance between the work machine 1A and the target surface in the MG can be ensured only by the coordinate values calculated by the work machine coordinate system coordinate calculator 21.
- the work machine coordinate system drawing image generation unit 22 can be omitted.
- the work machine coordinate system drawing image generation unit 22 generates the work machine 1A in the work machine coordinate system Co3 based on the coordinate values of the work machine corresponding points in the work machine coordinate system Co3 converted by the work machine coordinate system coordinate conversion unit 24.
- a drawing image (hereinafter, sometimes referred to as a “working machine coordinate system drawing image”, which includes, for example, an image of the bucket 4 as viewed from the side) is generated, and the drawing image is output to the work machine controller 50.
- a specific generation method of the drawn image of the work machine 1A for example, a region surrounded by a point on the contour of the work machine 1A in the work machine coordinate system Co3 output by the work machine coordinate system coordinate calculation unit 21 is described.
- a method of executing a process of painting with a predetermined color as the color of the work machine 1A is used in addition to the method of filling the internal area of the outline of the work machine 1A with a specific color. May be.
- the drawing image is created in this manner, the same image as the image shown on the side surface image of the working machine (that is, the real image) can be displayed on the display monitor 18, so that the occurrence of discomfort by the operator can be extremely easily suppressed.
- a method of deforming an image or the like prepared in advance according to the contour may be used.
- the measurement controller 20 (imaging device coordinate system coordinate conversion unit 24) of the present embodiment extracts the outline of the work machine 1A in the work machine side image by image processing, and extracts the contour in the work machine side image. Based on the position information of an arbitrary pixel on the line (for example, all the pixels on the contour line) and the equation of the plane S1, the coordinate value of the arbitrary pixel in the imaging device coordinate system Co1 of the work machine corresponding point of the arbitrary pixel is calculated. Thus, the measurement controller 20 can automatically acquire the position information of the outer shape (outline) when the work machine 1A is viewed from the side.
- the measurement controller 20 (working machine coordinate system drawing image generating unit 22) of the present embodiment converts the coordinate values of the working machine corresponding point in the working machine coordinate system Co3 converted by the working machine coordinate system coordinate conversion unit 25. , A drawing image of the work machine 1A in the work machine coordinate system Co3 can be generated. Thereby, since the outer shape of the drawn image of the work implement 1A displayed on the display monitor 18 approaches the real one, it is possible to prevent the image of the work implement 1A from being different from the real one and giving an uncomfortable feeling to the operator.
- the measurement controller 20 calculates the coordinate values in the photographing device coordinate system Co1 for the work machine corresponding points of all the pixels constituting the work machine 1A on the work machine side image (photographing device coordinate system coordinate conversion).
- Unit 24 by arranging (mapping) the same pixels as the pixels corresponding to the respective coordinate values, to generate a drawn image of the working machine 1A (working machine coordinate system drawn image generating unit 22), the display monitor 18
- the appearance of the displayed drawn image of the work machine 1A can be made closer to the real thing.
- the position of the plane S1 in the photographing device coordinate system Co1 is specified based on the side image of the work machine 1A to which the three known point markers 40 are attached. Not limited to this. For example, (1) three or more markers 40 are projected onto the side surface of the working machine 1A from a projection device such as a projector whose positional relationship with the photographing device 19 is known, and the markers 40 are used to photograph the working machine side images by the photographing device 19.
- a distance measuring device for example, a laser type, an LED type, and the like capable of measuring the distance between an arbitrary point on the side surface of the work machine 1A and the image capturing device 19 when the positional relationship with the capturing device 19 is known.
- Ultrasonic distance The sensor, may be used a method of calculating a plane equation S1 is to acquire the distance information over any three points on the side of the working machine 1A.
- the specification of the plane S1 is not limited to the method of specifying from three or more positions on the plane S1 as described above. For example, if the inclination (for example, the normal vector) of the plane S1 is known, the plane S1 can be specified only by the position of one point on the plane S1.
- FIG. 9 is a system configuration diagram of the excavator 1 of FIG.
- the hydraulic excavator 1 according to the present embodiment includes an engine 47, a hydraulic pump 46 and a pilot pump (not shown) mechanically connected to an output shaft of the engine 47 and driven by the engine 47, and a pressure discharged from the pilot pump.
- the operating levers 10 and 11 output the oil pressure reduced according to the operation amount to the control valve 45 via the proportional solenoid valve 39 as a control signal for each hydraulic actuator 5-9, and the hydraulic pump 46
- a plurality of control valves 45 for controlling the flow rate and direction of the hydraulic oil introduced into the control valve 9 based on control signals (pilot pressure) output from the operation levers 10 and 11 or the proportional solenoid valve 39;
- Pressure sensors 48 for detecting the pressure value of the pilot pressure acting on the
- a work machine controller 50 that calculates a corrected target pilot pressure based on the vehicle information and outputs a command voltage capable of generating the corrected target pilot pressure to the proportional solenoid valve 27, and a target surface formed by the work machine 1A.
- a target plane data input device 37 for inputting information to the work machine controller 50 is provided.
- the torque and flow rate of the hydraulic pump 46 are mechanically controlled so that the vehicle body operates according to the target output of each hydraulic actuator 5-8.
- control valves 45 There are the same number of control valves 45 as the number of hydraulic actuators 5-8 to be controlled, but FIG. 9 shows them as one.
- Each control valve is acted upon by two pilot pressures that move the internal spool in one or the other axial direction. For example, a boom raising pilot pressure and a boom lowering pilot pressure act on the control valve 45 for the boom cylinder 5.
- the pressure sensors 48 detect the pilot pressure acting on each control valve 45, and may have twice as many control valves.
- the pressure sensor 48 is provided immediately below the control valve 45, and detects a pilot pressure actually acting on the control valve 45.
- proportional solenoid valves 39 Although there are a plurality of proportional solenoid valves 39, they are collectively shown in one block in FIG. There are two types of proportional solenoid valves 39. One is a pressure reducing valve that reduces the pilot pressure input from the operating levers 10 and 11 to a desired corrected target pilot pressure specified as an output or a command voltage, and the other is a pressure reducing valve. This is a pressure increasing valve that reduces the pilot pressure input from the pilot pump to a desired corrected target pilot pressure specified by the command voltage and outputs the pilot pressure when a pilot pressure higher than the pilot pressure output by the controller 11 is required.
- the pilot pressure is generated via the pressure increasing valve and output from the operation levers 10 and 11.
- the pilot pressure is generated via a pressure reducing valve.
- the pilot pressure is not output from the operating levers 10 and 11, the pilot pressure is generated via a pressure increasing valve.
- the pressure reducing valve and the pressure increasing valve allow the pilot pressure having a pressure value different from the pilot pressure (the pilot pressure based on the operator's operation) input from the operating levers 10 and 11 to act on the control valve 45.
- the desired operation can be performed by the hydraulic actuator to be controlled.
- ⁇ ⁇ ⁇ There can be a maximum of two pressure reducing valves and two pressure increasing valves for one control valve 45.
- two pressure reducing valves and two pressure increasing valves are provided for the control valve 45 of the boom cylinder 5.
- a first pressure reducing valve provided in a first conduit for guiding the boom raising pilot pressure from the operation lever 11 to the control valve 45, and the boom raising pilot pressure bypassing the operation lever 11 from the pilot pump.
- the hydraulic excavator 1 includes a second pressure increasing valve provided in a fourth conduit for guiding the pilot pressure from the pilot pump to the control valve 45 by bypassing the operation lever 11.
- the work machine controller 50 includes a position and orientation detection unit 26, an information processing unit 30, a display control unit 33, and a work machine control unit 35.
- the position / posture detecting unit 26 includes a working machine posture detecting unit 27, a vehicle body position detecting unit 28, and a vehicle body angle detecting unit 29, and receives various kinds of sensor information as input, and acquires posture information, vehicle body position information, and vehicle body information of the working machine 1A. Outputs angle information.
- the work implement attitude detection unit 27 detects the attitude of the work implement 1A in the vehicle body coordinate system Co4 based on the outputs of the attitude sensors 12, 13, and 14 attached to the work implement 1A. More specifically, operations such as rotation angles ⁇ , ⁇ , and ⁇ (see FIG. 3) of the boom 2, the arm 3, and the bucket 4 based on information from the boom angle sensor 12, the arm angle sensor 13, and the bucket angle sensor 14. The attitude information of the machine 1A is detected.
- the vehicle body position detector 28 detects vehicle body position information based on information obtained by the first GNSS antenna 17a.
- the vehicle body angle detecting unit 29 determines the inclination angle ⁇ (see FIG. 3) by the vehicle body front-rear inclination sensor 16a, the left-right inclination angle ⁇ (not shown) by the vehicle body left-right inclination sensor 16b, and outputs the signal to the first GNSS antenna 17a.
- the azimuth angle of the vehicle body is detected from the position information of the 2GNSS antenna 17b to obtain vehicle body angle information.
- the vehicle body position information is obtained based on the information of the first GNSS antenna 17a.
- the position information of the second GNSS antenna 17b may be used, or a three-dimensional surveying instrument such as a total station may be used. May be.
- the azimuth information of the vehicle body is detected from the position information of the first GNSS antenna 17a and the second GNSS antenna 17b.
- a method using an electronic compass or a method using a turning angle sensor may be used.
- the information processing unit 30 includes a vehicle body coordinate conversion unit 31 and a target plane calculation unit 32.
- the input data of the information processing unit 30 include a work machine coordinate system coordinate value and a work machine coordinate system drawing image output from the measurement controller 20, target plane data input by the target plane data input device 37, and position and orientation detection.
- the output data of the information processing unit 30 includes, in addition to the posture information, the vehicle body position information, and the vehicle body angle information of the work machine 1A input from the position and posture detection unit 26, the work machine 1A
- the vehicle body coordinate conversion unit 31 outputs the coordinate values (vehicle body coordinates) of the work equipment corresponding points in the work equipment coordinate system Co3 output from the measurement controller 20 (work equipment coordinate system coordinate conversion unit 25 and work equipment coordinate system drawing image generation unit 22).
- the system coordinate value information) and the drawn image of the work machine 1A are converted into coordinate values in a vehicle body coordinate system Co4, which is a two-dimensional coordinate system set for the excavator 1.
- the working machine coordinate system coordinate values and the working machine coordinate system drawing image output from the measurement controller 20 are detected by the working machine posture detecting unit 27 of the position and posture detecting unit 26 as shown in FIG.
- the amount of translation and rotation for converting the work machine coordinate system coordinate value and the work machine coordinate system drawing image into the coordinate value of the vehicle body coordinate system Co4 are determined by the rotation angle ⁇ of the boom 2, arm 3, and bucket 4.
- ⁇ and ⁇ are known, they can be obtained by comparing the coordinate value of the work machine coordinate system Co3 with the coordinate value of the vehicle body coordinate system Co4 measured by a measuring device such as a total station for any two different points. Good.
- the target plane calculation unit 32 calculates a line segment where the target plane data (three-dimensional data) 51 input by the target plane data input device 37 and the XZ plane of the vehicle body coordinate system Co4 intersect.
- the line segment is set as the target plane 55.
- the XZ plane of the vehicle body coordinate system Co4 is obtained based on the vehicle body position information output by the position and orientation detection unit 26 and the vehicle body angle information output by the vehicle body angle detection unit 29.
- the target plane data 51 input by the target plane data input device 37 is assumed to be three-dimensional data, but may be two-dimensional data, that is, line segment data indicating the target plane. When the target plane data is two-dimensional data, it is not necessary to use the vehicle body position information of the vehicle body position detector 28 and the vehicle azimuth angle information of the vehicle body angle detector 29.
- the display control unit 33 displays the drawn image of the work implement 1A in the work machine coordinate system Co4 converted by the body coordinate conversion unit 31 and the work machine corresponding point converted in the body coordinate system Co4 by the body coordinate conversion unit 31. Based on the coordinate values and the posture of the work implement 1A in the vehicle body coordinate system Co4 obtained by the posture sensors 12, 13, and 14, the drawn image of the work implement 1A is adjusted to the posture of the work implement 1A in the vehicle body coordinate system Co4. It is displayed on the display monitor 18.
- the display control unit 33 includes a target plane information calculation unit 34.
- the posture information, the vehicle body position information and the vehicle body angle information of the work machine 1A output from the information processing unit 30, and the work machine 1A includes vehicle body coordinate system coordinate value information, vehicle body coordinate system drawing image information, and target plane information on the vehicle body coordinate system XZ plane.
- the output data includes work machine-target plane vector information in addition to the input information.
- the output information is input to the display monitor 18 and presented to the user.
- the target plane information calculation unit 34 will be described with reference to FIG. 12 showing an example of the positional relationship between the work implement 1A and the target plane 55.
- the target plane information calculation unit 34 determines, for an arbitrary point (point P in FIG. 12) on the outline of the work machine 1A, a plurality of planes (hereinafter, referred to as “target plane:”) constituting the target plane 55. ), The nearest points (points P1 and P2) on planes (target plane 1 and target plane 2 in FIG. 12) existing within a certain distance from the working machine 1A. Is obtained from the point (point P) to the nearest point (point P1 and point P2) on the target plane.
- the target plane vector (vector PP1 and vector PP2) is obtained.
- the output information is obtained from a point on the work machine 1A to the target plane 55.
- Plane Although the vector information up to the nearest point in the above is used, distance information may be output, or a vertical distance from a point on the work implement 1A to the target plane 55 may be output.
- the calculation is performed for all points on the contour line of the work machine 1A, the calculation may be performed only for specific points such as a point at the tip of the work machine 1A and a point on the back of the work machine 1A.
- FIG. 13 shows an example of a screen displayed on the display monitor 18.
- a vehicle body image IM1 drawn based on vehicle body coordinate system drawing image information
- a target plane image IM2 drawn based on target plane data on the XZ plane of the vehicle body coordinate system Co4
- a work The machine-work plane drawn based on the target plane vector information-the target plane vector image IM3 is displayed.
- the work implement-target plane vector image IM3 is obtained by drawing the work implement-target plane vector information output from the target plane information calculation unit 34 that has the smallest vector size for each target plane. is there. It is assumed that the magnitude of the vector when the point on the contour of the working machine is sunk into the target plane 55 has a negative value.
- the guidance screen IM may display information output from the target surface information calculation unit 34 and information obtained by processing the information, in addition to the example described in the present embodiment. Further, in the present embodiment, only the guidance screen IM displayed on the display monitor 18 as the machine guidance function has been described, but information may be presented by sound or vibration in addition to such visual information.
- the work machine control unit 35 determines the position information of the predetermined target plane 55 input from the information processing unit 30 and the coordinate values in the vehicle body coordinate system Co4 of the work machine corresponding points subjected to the coordinate transformation by the vehicle body coordinate transformation unit 31. Based on the posture of the work implement 1A in the vehicle body coordinate system Co4 obtained by the posture sensors 12, 13, and 14, the control point of the work implement 1A corresponding to the work implement corresponding point is held above the target plane 55. Work machine 1A (hydraulic cylinders 5, 6, 7) is controlled as described above.
- the work machine control unit 35 includes a target operation calculation unit 36.
- the input data of the work machine control unit 35 include the output of the display control unit 33, the posture information of the work machine 1A of the position and posture detection unit 26, the operation levers 10, 11, and 11.
- An operation input to the operation input device consisting of the following is made, and as the output data, there is a control signal of the proportional solenoid valve 39.
- the target operation calculating unit 36 predicts the moving direction and speed of the work machine 1A based on the input information (the posture information of the work machine 1A and the operation input information of the operation levers 10 and 11). At this time, for example, when it is predicted that the work implement 1A is sunk into the target surface 55, a control signal for decreasing or increasing the pilot pressure is transmitted to the electromagnetic proportional valve 39 so that the work implement 1A does not move into the target surface 55. Output to The pilot pressure corrected by the electromagnetic proportional valve 39 drives the control valve 45, and the hydraulic cylinders 5, 6, 7 are appropriately driven based on the operation to prevent the work machine 1 ⁇ / b> A from sinking into the target surface 55. You. In the present embodiment, the solenoid proportional valve 39 controls the pilot pressure, but the solenoid proportional valve may directly control the operating oil pressure of the actuator.
- the shape of the actual work machine 1A calculated by the measurement controller 20 based on the side image of the work machine 1A is used.
- MG and MC are performed using the coordinate information and the drawn image that match well.
- the accuracy of the guidance information for example, the work implement-target plane vector image IM3 and the distance information from the work implement 1A to the target plane 55
- the work machine control unit 35 can perform accurate MC even when the work machine 1A has a shape that requires many control points such as a curved portion and a protrusion.
- the photographing device 19 and the measurement controller 20 are mounted on the excavator 1 and measure the outer shape information of the working machine 1A (the body coordinate system coordinate value information and the body coordinate system drawing image information of the working machine 1A).
- the feature is that the machine guidance and machine control functions are provided while performing in real time.
- the same parts as those in the previous embodiment are denoted by the same reference numerals, and the description may be appropriately omitted.
- the imaging device 19 of the present embodiment is attached to the front of the upper revolving unit 1BA via a support device (articulated arm) 60.
- the support device 60 in FIG. 14 is a horizontal articulated arm formed by connecting a plurality of horizontal arms, and drives an actuator (for example, a motor) 19b embedded in each joint to drive the imaging device 19 in the vehicle body coordinate system Co4. The direction and position can be changed.
- each joint of the support device 60 is provided with an angle sensor (photographing device sensor) 19a for detecting the rotation angle of each horizontal arm, and the detection value of the angle sensor 19a is transmitted to the measurement controller 20 as shown in FIG. Has been output.
- the support device 60 is a horizontal articulated arm. However, an arm that can move in the vertical direction can be used, and another support device can be used.
- FIG. 15 is a system configuration diagram of the excavator 1 according to the present embodiment.
- the hydraulic shovel 1 of the present embodiment includes an imaging device 19, a measurement controller 20, and a work machine controller 50.
- the photographing device 19 photographs the work machine side images at predetermined intervals, and the measurement controller 20 uses the work machine 1A coordinate values and the body coordinate system of the work machine 1A in real time based on the work machine side images.
- the drawing image is calculated and output to the work machine controller 50.
- the measurement controller 20 of the first embodiment outputs the coordinate values and the drawn image in the working machine coordinate system Co3
- the photographing device 19 is attached to the vehicle body (the upper swing body 1BA) of the excavator 1. Therefore, the coordinate value and the drawn image in the vehicle body coordinate system Co4 can be directly calculated.
- the work machine controller 50 of the present embodiment provides a user with machine guidance and a machine control function based on information output from the measurement controller 20 in real time.
- the measurement controller 20 includes a vehicle body coordinate system coordinate calculation unit 21b and a vehicle body coordinate system drawing image generation unit 22b.
- the position information and the orientation information of the photographing device 19 in the vehicle body coordinate system are input to the measurement controller 20 from the side image of the work machine 1A photographed by the photographing device 19 and the angle sensor 19a.
- the vehicle body coordinate system coordinate calculator 21b includes a photographing position calculator 23, a photographing device coordinate system coordinate converter 24, and a vehicle body coordinate system coordinate converter 25b.
- a side image is input, a body coordinate system coordinate value is output for a designated point on the side image of the work machine 1A, and a body coordinate system drawing that matches the shape and dimensions of the work machine 1A in the body coordinate system Co4.
- the body coordinate system coordinate values of the working machine corresponding points of all the pixels on the contour of the working machine in the working machine side image are output.
- other methods For example, it is needless to say that only the coordinate values of the working machine corresponding points of some pixels on the contour line may be used.
- the shooting position calculation unit 23 and the shooting device coordinate system coordinate conversion unit 24 perform the same calculation as in the first embodiment.
- the body coordinate system coordinate conversion unit 25b translates and rotates the coordinate values in the photographing device coordinate system Co1 based on the position information and orientation information in the vehicle body coordinate system Co4 of the photographing device 19 input from the angle sensor 19a.
- the coordinate is converted into a coordinate value in the vehicle body coordinate system Co4.
- the vehicle body coordinate system drawing image generation unit 22b generates a drawing image after the coordinate conversion to the vehicle body coordinate system Co4.
- the information (the vehicle body coordinate system coordinate value and the vehicle body coordinate system drawing image of the work machine 1A) input from the measurement controller 20 to the work machine controller 50 is already information in the vehicle body coordinate system Co4.
- the information processing unit 30 of the work machine controller 50 of the present embodiment does not include the vehicle body coordinate conversion unit 31 of the first embodiment, but the other parts have the same configuration and the same processing.
- the work machine posture detection unit 27 in the position and posture detection unit 26 is unnecessary because the measurement controller 20 can detect the posture of the work machine 1A.
- the processing contents of the display control unit 33 and the work implement control unit 35 are the same as in the first embodiment.
- the measurement controller 20 measures the position and the shape information of the work machine 1A in real time. Therefore, the user does not need to measure the shape and the like of the work machine 1A in advance as in the first embodiment, and can easily obtain work machine shape information. Further, since the position and shape of the work machine 1A are acquired in real time, even when the work machine 1A is worn or deformed, the work machine shape can be accurately measured. Can be presented in a way that is easy for the user to understand. In the work implement control unit 35, accurate control can be performed according to the actual state of the work implement.
- the present invention is not limited to the above embodiment, and includes various modifications without departing from the gist of the present invention.
- the present invention is not limited to one having all the configurations described in the above embodiment, but also includes one in which a part of the configuration is deleted. Further, a part of the configuration according to one embodiment can be added to or replaced by the configuration according to another embodiment.
- the components related to the controllers 20 and 50 and the functions and execution processes of the components are partially or wholly implemented by hardware (for example, a logic that executes each function is designed by an integrated circuit). May be realized.
- the configuration of the controllers 20 and 50 may be a program (software) that realizes each function of the configuration of the controllers 20 and 50 by being read and executed by an arithmetic processing unit (for example, a CPU).
- Information relating to the program can be stored in, for example, a semiconductor memory (flash memory, SSD, etc.), a magnetic storage device (hard disk drive, etc.), a recording medium (magnetic disk, optical disk, etc.), and the like.
- SYMBOLS 1 Hydraulic excavator (work machine), 1A ... Work machine (front work machine), 1B ... Body, 1BA ... Upper revolving body, 1BB ... Lower traveling body, 2 ... Boom, 3 ... Arm, 4 ... Bucket, 5 ... Boom Cylinder, 6: Arm cylinder, 7: Bucket cylinder, 10, 11: Operation lever, 12: Boom angle sensor (posture sensor), 13: Arm angle sensor (posture sensor), 14: Bucket angle sensor (posture sensor), 18 ... Display monitor (display device), 19 ... Imaging device, 20 ... Measurement controller, 21 ... Work machine coordinate system coordinate calculation unit, 22 ... Work machine coordinate system drawing image generation unit, 23 ... Shooting position calculation unit, 24 ... Shooting device Coordinate system coordinate converter, 25: work machine coordinate system coordinate converter, 39: proportional solenoid valve, 40: known point marker, 50: work machine controller
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Civil Engineering (AREA)
- Structural Engineering (AREA)
- Mining & Mineral Resources (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- Manufacturing & Machinery (AREA)
- Automation & Control Theory (AREA)
- Component Parts Of Construction Machinery (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
第1実施形態では,油圧ショベル(作業機械)1に搭載された作業機1Aを撮影する撮影装置(例えばカメラ)19と,撮像装置19により作業機1Aの側面を撮影した画像(以下では「作業機側面画像」と称することがある)を利用して,作業機1Aの形状に関する情報を計測する測定コントローラ20と,油圧ショベル1に搭載され,測定コントローラ20で演算された作業機1Aの形状に関する情報を入力して油圧ショベル1で実行される例えばMGやMCに利用する作業機械コントローラ50とを備えるシステムについて説明する。
図4は本発明の実施形態に係る測定コントローラ20の機能ブロック図である。この図に示すように,測定コントローラ20は,作業機座標系Co3での作業機1Aの座標値を演算する作業機座標系座標演算部21と,作業機座標系Co3での作業機1Aの描画画像を生成する作業機座標系描画画像生成部22とを備えており,撮影装置19で撮影された作業機1Aの側面写真の入力を受けている。
(1)以上のような撮影装置19及び測定コントローラ20で構成された測定システムによれば,作業機1A(例えばバケット4)の外形情報を取得するに際して,ユーザーは作業機1Aの側面に3つ以上の既知点マーカー40を取り付け,その画像(作業機側面画像)を撮影装置19で撮影する操作のみを行えば良い。作業機側面画像の撮影後は,測定コントローラ20が,複数の既知点マーカー40によって定義される平面S1の方程式を作業機側面画像と撮像装置18の内部パラメータに基づいて演算し(撮影位置演算部23による処理),作業機側面画像上で作業機1Aの輪郭線上に位置する全ての画素(作業機構成画素)の位置情報と平面S1の方程式とに基づいて当該全ての画素の作業機対応点の撮影装置座標系Co1における座標値を算出し(撮影装置座標系座標変換部24による処理),その座標値を作業機座標系Co3の座標値に変換して作業機械コントローラ50に出力する(作業機座標系座標変換部25による処理)。これにより作業機側面画像における作業機1Aの輪郭線上に位置する全ての画素の作業機対応点の位置情報を容易に取得することができるので,作業機械1Aの正確な外形情報を従来に比して簡単に測定できる。その結果,作業機1Aの実際の形状に即したMCやMGが実行されることになり,その正確度が向上するので作業効率の向上が見込める。
次に測定コントローラ20から出力された作業機1Aの座標値と描画画像の作業機械コントローラ50での利用について説明する。
作業機械コントローラ50は,位置姿勢検出部26と,情報処理部30と,表示制御部33と,作業機制御部35を備えている。
第2実施形態では,撮影装置19と測定コントローラ20を油圧ショベル1に搭載しており,作業機1Aの外形情報(作業機1Aの車体座標系座標値情報及び車体座標系描画画像情報)の計測をリアルタイムに行いながらマシンガイダンスやマシンコントロール機能を提供している点に特徴がある。なお,先の実施形態と同じ部分には同じ符号を付して説明を適宜省略することがある。
Claims (13)
- 作業機械に備えられた作業機の外形形状を測定する測定コントローラを備える作業機の外形形状測定システムにおいて,
前記作業機の側面を撮影する撮影装置を備え,
前記測定コントローラは,
前記撮影装置により前記作業機の側面を撮影した画像と前記撮影装置の内部パラメータとに基づいて,前記撮影装置に設定された3次元座標系である撮影装置座標系において前記作業機の側面を表す平面の位置を算出し,
前記画像上で前記作業機を構成する任意の画素の前記画像における位置情報と前記平面の位置とに基づいて前記画素に対応する前記作業機上の点の前記撮影装置座標系における座標値を算出し,
前記画素に対応する前記作業機上の点の前記撮影装置座標系における座標値を前記作業機に設定された2次元座標系である作業機座標系における座標値に変換して前記作業機械の作業機械コントローラに出力することを特徴とする作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムにおいて,
前記測定コントローラは,前記画像における前記作業機の輪郭線を抽出し,前記画像における前記輪郭線上の任意の画素の位置情報と前記平面の位置とに基づいて,前記画素に対応する前記作業機上の点の前記撮影装置座標系における座標値を算出することを特徴とする作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムにおいて,
前記測定コントローラは,前記画像における前記作業機の輪郭線を抽出し,前記画像における前記輪郭線上のすべての画素の位置情報と前記平面の位置とに基づいて,前記すべての画素に対応する前記作業機上の点の前記撮影装置座標系における座標値を算出することを特徴とする作業機械の作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムにおいて,
前記画素に対応する前記作業機上の点の前記撮影装置座標系における座標値は,前記画像上で前記作業機を構成するすべての画素に対応する前記作業機上の複数の点について演算されることを特徴とする作業機械の作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムにおいて,
前記測定コントローラは,前記作業機の側面に取り付けられ互いの距離が既知の3つ以上のマーカーの前記画像における画素位置に基づいて前記平面の位置を算出することを特徴とする作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムにおいて,
前記撮影装置との位置関係が既知な投影装置をさらに備え,
前記画像には,前記投影装置から前記作業機の側面に投影されたマーカーが撮影されており,
前記測定コントローラは,前記画像における前記マーカーの画素位置に基づいて前記平面の位置を算出することを特徴とする作業機の外形形状測定システム。 - 請求項4の作業機の外形形状測定システムにおいて,
前記撮影装置は,互いの位置関係が既知な複数の撮影装置であり,
前記測定コントローラは,前記複数の撮影装置間の距離をもとに前記平面の位置を算出することを特徴とする作業機の外形形状測定システム。 - 請求項4の作業機の外形形状測定システムにおいて,
前記撮影装置と前記作業機の側面上の任意の点との距離を測る測距装置をさらに備え,
前記測定コントローラは,前記測距装置により測定された前記作業機の側面上の3点以上の距離情報をもとに前記平面の位置を算出することを特徴とする作業機械の作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムにおいて,
前記測定コントローラは,前記画素に対応する前記作業機上の点の前記作業機座標系における座標値をもとに前記作業機座標系における前記作業機の描画画像を生成することを特徴とする作業機の外形形状測定システム。 - 請求項1の作業機の外形形状測定システムと,前記作業機械コントローラと,前記作業機械に搭載された表示装置とを備えた作業機の外形形状表示システムにおいて,
前記測定コントローラは,前記画素に対応する前記作業機上の点の前記作業機座標系における座標値をもとに前記作業機座標系における前記作業機の描画画像を生成し,
前記作業機械コントローラは,
前記測定コントローラから出力される前記画素に対応する前記作業機上の点の前記作業機座標系における座標値を前記作業機械に設定された2次元座標系である車体座標系における座標値に変換し,
前記作業機に取り付けられた姿勢センサの出力に基づいて前記車体座標系における前記作業機の姿勢を検出し,
前記作業機座標系における前記作業機の描画画像と,前記画素に対応する前記作業機上の点の前記車体座標系における座標値と,前記車体座標系における前記作業機の姿勢とに基づいて,前記車体座標系における前記作業機の姿勢に合わせて前記作業機の描画画像を前記表示装置に表示することを特徴とする作業機の外形形状表示システム。 - 請求項1の作業機の外形形状測定システムと,前記作業機械コントローラとを備えた作業機の制御システムにおいて,
前記作業機械コントローラは,
前記測定コントローラから出力される前記画素に対応する前記作業機上の点の前記作業機座標系における座標値を前記作業機械に設定された2次元座標系である車体座標系における座標値に変換し,
前記作業機に取り付けられた姿勢センサの出力に基づいて前記車体座標系における前記作業機の姿勢を検出し,
予め定められた目標面の位置情報と,前記画素に対応する前記作業機上の点の前記車体座標系における座標値と,前記車体座標系における前記作業機の姿勢とに基づいて,前記画素に対応する前記作業機のコントロールポイントが前記目標面の上方に保持されるように前記作業機を制御することを特徴とする作業機の制御システム。 - 作業機と,表示装置と,予め定められた目標面と前記作業機の位置関係を前記表示装置に表示させる作業機械コントローラとを備えた作業機械において,
前記作業機の側面を撮影する撮影装置と,
前記撮影装置により前記作業機の側面を撮影した画像と前記撮影装置の内部パラメータとに基づいて,前記撮影装置に設定された3次元座標系である撮影装置座標系において前記作業機の側面を表す平面の位置を算出し,前記画像上で前記作業機を構成する任意の画素の前記画像における位置情報と前記平面の位置とに基づいて前記画素に対応する前記作業機上の点の前記撮影装置座標系における座標値を算出し,前記画素に対応する前記作業機上の点の前記撮影装置座標系における座標値を前記作業機械の車体に設定された2次元座標系である車体座標系における座標値に変換して前記作業機械コントローラに出力し,前記画素に対応する前記作業機上の点の前記車体座標系における座標値をもとに前記車体座標系における前記作業機の描画画像を生成して前記作業機械コントローラに出力する測定コントローラとを備え,
前記作業機械コントローラは,
前記作業機に取り付けられた姿勢センサの出力に基づいて前記車体座標系における前記作業機の姿勢を検出し,
前記車体座標系における前記作業機の描画画像と,前記画素に対応する前記作業機上の点の前記車体座標系における座標値と,前記車体座標系における前記作業機の姿勢とに基づいて,前記車体座標系における前記作業機の姿勢に合わせて前記作業機の描画画像を前記表示装置に表示する
ことを特徴とする作業機械。 - 請求項12の作業機械において,
前記作業機械コントローラは,予め定められた目標面の位置情報と,前記画素に対応する前記作業機上の点の前記車体座標系における座標値と,前記車体座標系における前記作業機の姿勢とに基づいて,前記画素に対応する前記作業機上の点に対応する前記作業機のコントロールポイントが前記目標面の上方に保持されるように前記作業機を制御する
ことを特徴とする作業機械。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2018/035525 WO2020065738A1 (ja) | 2018-09-25 | 2018-09-25 | 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 |
KR1020207004801A KR102231510B1 (ko) | 2018-09-25 | 2018-09-25 | 작업기의 외형 형상 측정 시스템, 작업기의 외형 형상 표시 시스템, 작업기의 제어 시스템 및 작업 기계 |
US16/644,229 US11434623B2 (en) | 2018-09-25 | 2018-09-25 | Work-implement external-shape measurement system, work-implement external-shape display system, work-implement control system and work machine |
JP2020509531A JP6826233B2 (ja) | 2018-09-25 | 2018-09-25 | 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 |
CN201880055028.1A CN111226009B (zh) | 2018-09-25 | 2018-09-25 | 作业机的外形形状测定系统、作业机的外形形状显示系统、作业机的控制系统以及作业机械 |
EP18931280.4A EP3859090A4 (en) | 2018-09-25 | 2018-09-25 | OUTER PROFILE MEASUREMENT SYSTEM FOR EARTHWORKING MACHINE, OUTER PROFILE DISPLAY SYSTEM FOR EARTHWORKING MACHINE, CONTROL SYSTEM FOR EARTHWORKING MACHINE, AND EARTHWORKING MACHINE |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2018/035525 WO2020065738A1 (ja) | 2018-09-25 | 2018-09-25 | 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020065738A1 true WO2020065738A1 (ja) | 2020-04-02 |
Family
ID=69952994
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2018/035525 WO2020065738A1 (ja) | 2018-09-25 | 2018-09-25 | 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 |
Country Status (6)
Country | Link |
---|---|
US (1) | US11434623B2 (ja) |
EP (1) | EP3859090A4 (ja) |
JP (1) | JP6826233B2 (ja) |
KR (1) | KR102231510B1 (ja) |
CN (1) | CN111226009B (ja) |
WO (1) | WO2020065738A1 (ja) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7155516B2 (ja) * | 2017-12-20 | 2022-10-19 | コベルコ建機株式会社 | 建設機械 |
CN112218988B (zh) * | 2018-07-31 | 2023-06-09 | 住友建机株式会社 | 挖土机 |
JP7166108B2 (ja) * | 2018-08-31 | 2022-11-07 | 株式会社小松製作所 | 画像処理システム、表示装置、画像処理方法、学習済みモデルの生成方法、および学習用データセット |
JP7301514B2 (ja) * | 2018-09-21 | 2023-07-03 | 日立建機株式会社 | 座標変換システム及び作業機械 |
CN109782767B (zh) * | 2019-01-25 | 2022-06-07 | 北京百度网讯科技有限公司 | 用于输出信息的方法和装置 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005114235A (ja) * | 2003-10-07 | 2005-04-28 | Hitachi Constr Mach Co Ltd | 移動式作業機の埋設物処理支援装置 |
WO2016056674A1 (ja) | 2015-10-23 | 2016-04-14 | 株式会社小松製作所 | 作業機械の表示システム、作業機械及び表示方法 |
JP2016089388A (ja) * | 2014-10-30 | 2016-05-23 | 日立建機株式会社 | 作業支援画像生成装置、及びそれを備えた作業機械の遠隔操縦システム |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5476450B1 (ja) * | 2012-11-19 | 2014-04-23 | 株式会社小松製作所 | 掘削機械の表示システム及び掘削機械 |
US20150168136A1 (en) * | 2013-12-12 | 2015-06-18 | The Regents Of The University Of Michigan | Estimating three-dimensional position and orientation of articulated machine using one or more image-capturing devices and one or more markers |
EP3222042B1 (en) * | 2014-11-17 | 2022-07-27 | Yanmar Power Technology Co., Ltd. | Display system for remote control of working machine |
WO2016196292A1 (en) * | 2015-05-29 | 2016-12-08 | Hexagon Metrology, Inc. | Coordinate measuring machine with object location logic |
JP6322612B2 (ja) * | 2015-10-05 | 2018-05-09 | 株式会社小松製作所 | 施工管理システム及び形状計測方法 |
JP6925775B2 (ja) * | 2015-10-05 | 2021-08-25 | 株式会社小松製作所 | 施工管理システム |
JP6546558B2 (ja) * | 2016-03-31 | 2019-07-17 | 日立建機株式会社 | 建設機械及び建設機械の較正方法 |
JP6674846B2 (ja) * | 2016-05-31 | 2020-04-01 | 株式会社小松製作所 | 形状計測システム、作業機械及び形状計測方法 |
CA3029812C (en) * | 2016-08-31 | 2020-07-14 | Komatsu Ltd. | Image display system of work machine, remote operation system of work machine, work machine, and method for displaying image of work machine |
JP6867132B2 (ja) * | 2016-09-30 | 2021-04-28 | 株式会社小松製作所 | 作業機械の検出処理装置及び作業機械の検出処理方法 |
JP6496338B2 (ja) * | 2017-03-14 | 2019-04-03 | ファナック株式会社 | 工作機械の制御システム |
JP6445070B2 (ja) * | 2017-03-27 | 2018-12-26 | ファナック株式会社 | 工作機械の制御システム |
GB2563262B (en) * | 2017-06-08 | 2020-06-10 | Caterpillar Sarl | Improvements in the stability of work machines |
US10262411B2 (en) * | 2017-09-01 | 2019-04-16 | Deere & Company | Site scanning using a work machine with a camera |
JP6927821B2 (ja) * | 2017-09-15 | 2021-09-01 | 株式会社小松製作所 | 表示システム、及び表示装置 |
-
2018
- 2018-09-25 CN CN201880055028.1A patent/CN111226009B/zh active Active
- 2018-09-25 JP JP2020509531A patent/JP6826233B2/ja active Active
- 2018-09-25 KR KR1020207004801A patent/KR102231510B1/ko active IP Right Grant
- 2018-09-25 US US16/644,229 patent/US11434623B2/en active Active
- 2018-09-25 EP EP18931280.4A patent/EP3859090A4/en active Pending
- 2018-09-25 WO PCT/JP2018/035525 patent/WO2020065738A1/ja unknown
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005114235A (ja) * | 2003-10-07 | 2005-04-28 | Hitachi Constr Mach Co Ltd | 移動式作業機の埋設物処理支援装置 |
JP2016089388A (ja) * | 2014-10-30 | 2016-05-23 | 日立建機株式会社 | 作業支援画像生成装置、及びそれを備えた作業機械の遠隔操縦システム |
WO2016056674A1 (ja) | 2015-10-23 | 2016-04-14 | 株式会社小松製作所 | 作業機械の表示システム、作業機械及び表示方法 |
Also Published As
Publication number | Publication date |
---|---|
EP3859090A4 (en) | 2022-05-18 |
US20210156121A1 (en) | 2021-05-27 |
US11434623B2 (en) | 2022-09-06 |
KR20200037285A (ko) | 2020-04-08 |
CN111226009B (zh) | 2022-03-04 |
JPWO2020065738A1 (ja) | 2021-01-07 |
JP6826233B2 (ja) | 2021-02-03 |
KR102231510B1 (ko) | 2021-03-24 |
EP3859090A1 (en) | 2021-08-04 |
CN111226009A (zh) | 2020-06-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020065738A1 (ja) | 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 | |
JP7263127B2 (ja) | マップ生成システム及びマップ生成方法 | |
JP7301514B2 (ja) | 座標変換システム及び作業機械 | |
US11427988B2 (en) | Display control device and display control method | |
CN106029994B (zh) | 校正系统、作业机械和校正方法 | |
JP2018035645A (ja) | 作業機械の画像表示システム | |
JP2016160741A (ja) | 作業機械の画像表示システム、作業機械の遠隔操作システム及び作業機械 | |
JP6966218B2 (ja) | 撮像装置の校正装置、作業機械および校正方法 | |
JP7080750B2 (ja) | 表示制御システム、遠隔操作システム、表示制御装置、および表示制御方法 | |
JPWO2016047807A1 (ja) | 校正システム、作業機械及び校正方法 | |
JPWO2018043299A1 (ja) | 作業機械の画像表示システム、作業機械の遠隔操作システム、作業機械及び作業機械の画像表示方法 | |
JP2024028464A (ja) | 表示制御システムおよび表示制御方法 | |
JP2024052764A (ja) | 表示制御装置及び表示方法 | |
JP2017193958A (ja) | 校正システム、作業機械及び校正方法 | |
JP7536580B2 (ja) | 表示制御装置及び表示方法 | |
JP7128497B2 (ja) | 作業機械の画像表示システム | |
JP7333551B2 (ja) | 作業機械の画像表示システム | |
US11908076B2 (en) | Display system and display method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2020509531 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20207004801 Country of ref document: KR Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18931280 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2018931280 Country of ref document: EP Effective date: 20210426 |