WO2016148309A1 - 校正システム、作業機械及び校正方法 - Google Patents
校正システム、作業機械及び校正方法 Download PDFInfo
- Publication number
- WO2016148309A1 WO2016148309A1 PCT/JP2016/060273 JP2016060273W WO2016148309A1 WO 2016148309 A1 WO2016148309 A1 WO 2016148309A1 JP 2016060273 W JP2016060273 W JP 2016060273W WO 2016148309 A1 WO2016148309 A1 WO 2016148309A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pair
- information
- imaging devices
- target
- work machine
- Prior art date
Links
Images
Classifications
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/20—Drives; Control devices
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/261—Surveying the work-site to be treated
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/264—Sensors and their calibration for indicating the position of the work tool
- E02F9/265—Sensors and their calibration for indicating the position of the work tool with follow-up actions (e.g. control signals sent to actuate the work tool)
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
- B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F3/00—Dredgers; Soil-shifting machines
- E02F3/04—Dredgers; Soil-shifting machines mechanically-driven
- E02F3/28—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets
- E02F3/30—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets with a dipper-arm pivoted on a cantilever beam, i.e. boom
- E02F3/32—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets with a dipper-arm pivoted on a cantilever beam, i.e. boom working downwardly and towards the machine, e.g. with backhoes
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/20—Drives; Control devices
- E02F9/22—Hydraulic or pneumatic drives
- E02F9/2203—Arrangements for controlling the attitude of actuators, e.g. speed, floating function
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/24—Safety devices, e.g. for preventing overload
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/261—Surveying the work-site to be treated
- E02F9/262—Surveying the work-site to be treated with follow-up actions to control the work tool, e.g. controller
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/264—Sensors and their calibration for indicating the position of the work tool
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
Definitions
- the present invention relates to a calibration system, a work machine, and a calibration method for calibrating a position detection unit that is provided in a work machine and detects a target position.
- a work machine including an imaging device used for stereo three-dimensional measurement (for example, Patent Document 1).
- An imaging device used for stereo three-dimensional measurement needs to be calibrated.
- a work machine equipped with an imaging device calibrates the imaging device before being shipped from the factory.
- this calibration requires equipment and equipment, the imaging device can be calibrated at the work site. It can be difficult.
- An aspect of the present invention aims to realize calibration of an imaging apparatus even at a work site of a work machine including an imaging apparatus that performs three-dimensional measurement by a stereo method.
- a work machine having a work machine, at least a pair of imaging devices that pick up an image of the object, a position detector that detects a position of the work machine, and at least a pair of the above
- the position detector based on first position information that is information related to a predetermined position of the working machine imaged by the imaging device and an attitude of the working machine when at least a pair of the imaging devices image the predetermined position.
- Second position information that is information related to the predetermined position detected by the second position information
- third position information that is information related to the predetermined position outside the work machine captured by the pair of imaging devices.
- a work machine including the work machine and the calibration system according to the first aspect is provided.
- At least a pair of imaging devices captures a predetermined position of a work machine and a predetermined position around a work machine having the work machine, and at least a pair of the imaging devices.
- a detection step of detecting a predetermined position of the work machine with different position detectors, first position information that is information relating to a predetermined position of the work implement imaged by at least a pair of the imaging devices, and at least a pair of the above-mentioned The position of the work implement when the imaging device captures the predetermined position, and the second position information that is information related to the predetermined position detected by the position detector, and at least a pair of the imaging devices.
- a calibration method is provided.
- the present invention obtains conversion information for converting the position information of the object detected by the means for detecting the position of the object provided in the work machine into a coordinate system other than the means for detecting the position of the object. it can.
- the aspect of the present invention it is possible to realize the calibration of the imaging apparatus even at the work site of the work machine including the imaging apparatus that performs stereo three-dimensional measurement.
- FIG. 1 is a perspective view of a hydraulic excavator 100 including a calibration system according to an embodiment.
- FIG. 2 is a perspective view of the vicinity of the driver's seat of the excavator 100 according to the embodiment.
- FIG. 3 is a diagram illustrating dimensions of the working machine 2 included in the hydraulic excavator according to the embodiment and a coordinate system of the hydraulic excavator 100.
- a hydraulic excavator 100 that is a work machine has a vehicle body 1 and a work machine 2.
- the vehicle body 1 includes a revolving body 3, a cab 4, and a traveling body 5.
- the turning body 3 is attached to the traveling body 5 so as to be turnable.
- the cab 4 is disposed in the front part of the revolving structure 3.
- An operation device 25 shown in FIG. 2 is arranged in the cab 4.
- the traveling body 5 has crawler belts 5a and 5b, and the excavator 100 travels as the crawler belts 5a and 5b rotate.
- the work machine 2 is attached to the front part of the vehicle body 1.
- the work machine 2 includes a boom 6, an arm 7, a bucket 8 as a work tool, a boom cylinder 10, an arm cylinder 11, and a bucket cylinder 12.
- the front side of the vehicle body 1 is the direction side from the backrest 4SS of the driver's seat 4S shown in FIG.
- the rear side of the vehicle body 1 is the direction side from the operation device 25 toward the backrest 4SS of the driver's seat 4S.
- the front portion of the vehicle body 1 is a portion on the front side of the vehicle body 1 and is a portion on the opposite side of the counterweight WT of the vehicle body 1.
- the operating device 25 is a device for operating the work implement 2 and the swing body 3 and includes a right lever 25R and a left lever 25L.
- a monitor panel 26 is provided in the cab 4 in front of the driver's seat 4S.
- the base end portion of the boom 6 is attached to the front portion of the vehicle body 1 via a boom pin 13.
- the boom pin 13 corresponds to the operation center of the boom 6 with respect to the swing body 3.
- the proximal end portion of the arm 7 is attached to the distal end portion of the boom 6 via an arm pin 14.
- the arm pin 14 corresponds to the operation center of the arm 7 with respect to the boom 6.
- a bucket 8 is attached to the tip of the arm 7 via a bucket pin 15.
- the bucket pin 15 corresponds to the operation center of the bucket 8 with respect to the arm 7.
- the length of the boom 6, that is, the length between the boom pin 13 and the arm pin 14 is L1.
- the length of the arm 7, that is, the length between the arm pin 14 and the bucket pin 15 is L2.
- the length of the bucket 8, that is, the length between the bucket pin 15 and the blade tip P3 that is the tip of the blade 9 of the bucket 8 is L3.
- the boom cylinder 10, the arm cylinder 11 and the bucket cylinder 12 shown in FIG. These are actuators that are provided in the vehicle body 1 of the excavator 100 and operate the work implement 2.
- the base end portion of the boom cylinder 10 is attached to the revolving structure 3 via a boom cylinder foot pin 10a.
- the tip of the boom cylinder 10 is attached to the boom 6 via a boom cylinder top pin 10b.
- the boom cylinder 10 operates the boom 6 by expanding and contracting by hydraulic pressure.
- the base end of the arm cylinder 11 is attached to the boom 6 via an arm cylinder foot pin 11a.
- the tip of the arm cylinder 11 is attached to the arm 7 via an arm cylinder top pin 11b.
- the arm cylinder 11 operates the arm 7 by expanding and contracting by hydraulic pressure.
- the base end of the bucket cylinder 12 is attached to the arm 7 via a bucket cylinder foot pin 12a.
- the tip of the bucket cylinder 12 is attached to one end of the first link member 47 and one end of the second link member 48 via the bucket cylinder top pin 12b.
- the other end of the first link member 47 is attached to the tip of the arm 7 via a first link pin 47a.
- the other end of the second link member 48 is attached to the bucket 8 via a second link pin 48a.
- the bucket cylinder 12 operates the bucket 8 by expanding and contracting by hydraulic pressure.
- the boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12 are provided with a first angle detector 18A, a second angle detector 18B, and a third angle detector 18C, respectively.
- the first angle detector 18A, the second angle detector 18B, and the third angle detector 18C are, for example, stroke sensors. Each of them detects the stroke length of the boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12, so that the operating angle of the boom 6 with respect to the vehicle body 1, the operating angle of the arm 7 with respect to the boom 6, The operating angle of the bucket 8 is indirectly detected.
- the first angle detection unit 18A detects the operation amount of the boom cylinder 10, that is, the stroke length.
- the processing device 20 to be described later determines the boom 6 with respect to the Zm axis of the coordinate system (Xm, Ym, Zm) of the excavator 100 shown in FIG. 3 based on the stroke length of the boom cylinder 10 detected by the first angle detector 18A.
- the operating angle ⁇ 1 is calculated.
- the coordinate system of the excavator 100 is appropriately referred to as a vehicle body coordinate system. As shown in FIG. 2, the origin of the vehicle body coordinate system is the center of the boom pin 13.
- the center of the boom pin 13 is the center of the cross section when the boom pin 13 is cut on a plane orthogonal to the direction in which the boom pin 13 extends, and the center in the direction in which the boom pin 13 extends.
- the vehicle body coordinate system is not limited to the example of the embodiment.
- the turning center of the revolving structure 3 is the Zm axis
- the axis parallel to the direction in which the boom pin 13 extends is the Ym axis
- the axis may be the Xm axis.
- the second angle detector 18B detects the amount of movement of the arm cylinder 11, that is, the stroke length.
- the processing device 20 calculates the operating angle ⁇ 2 of the arm 7 relative to the boom 6 from the stroke length of the arm cylinder 11 detected by the second angle detector 18B.
- the third angle detector 18C detects the amount of movement of the bucket cylinder 12, that is, the stroke length.
- the processing device 20 calculates the operating angle ⁇ 3 of the bucket 8 relative to the arm 7 from the stroke length of the bucket cylinder 12 detected by the third angle detector 18C.
- the excavator 100 includes, for example, a plurality of imaging devices 30 a, 30 b, 30 c, and 30 d in the cab 4.
- the imaging device 30 when the plurality of imaging devices 30a, 30b, 30c, and 30d are not distinguished, they are appropriately referred to as the imaging device 30.
- the kind of the imaging device 30 is not limited, in the embodiment, for example, an imaging device including a CCD (Couple Charged Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor is used.
- a plurality of, more specifically, four imaging devices 30a, 30b, 30c, and 30d are attached to the excavator 100. More specifically, as illustrated in FIG. 2, the imaging device 30 a and the imaging device 30 b are disposed in the cab 4, for example, facing the same direction at a predetermined interval. The imaging device 30c and the imaging device 30d are arranged in the cab 4 with a predetermined interval and facing the same direction. The imaging device 30b and the imaging device 30d may be arranged slightly toward the work machine 2, that is, slightly toward the imaging device 30a and the imaging device 30c. A plurality of imaging devices 30a, 30b, 30c, and 30d are combined to form a stereo camera. In the embodiment, a stereo camera is configured by a combination of the imaging devices 30a and 30b and a combination of the imaging devices 30c and 30d.
- the excavator 100 includes the four image pickup devices 30, but the number of the image pickup devices 30 included in the excavator 100 may be at least two, that is, a pair, and is not limited to four. This is because the excavator 100 configures a stereo camera with at least a pair of the imaging devices 30 to capture the subject in stereo.
- the plurality of imaging devices 30 a, 30 b, 30 c, and 30 d are arranged in front of and above the cab 4.
- the upward direction is a direction perpendicular to the ground contact surfaces of the crawler belts 5a, 5b of the excavator 100 and away from the ground contact surfaces.
- the ground contact surfaces of the crawler belts 5a and 5b are planes defined by at least three points that do not exist on the same straight line at a portion where at least one of the crawler belts 5a and 5b is grounded.
- the plurality of imaging devices 30 a, 30 b, 30 c, and 30 d shoots a subject existing in front of the vehicle body 1 of the excavator 100 in stereo.
- the target is, for example, a target excavated by the work machine 2.
- the processing device 20 shown in FIG. 1 and FIG. 2 measures a target three-dimensionally using at least the result of stereo shooting by the pair of imaging devices 30. In other words, the processing device 20 performs stereo image processing on the same target image captured by at least the pair of imaging devices 30 to measure the above-described target three-dimensionally.
- the place where the plurality of imaging devices 30 a, 30 b, 30 c, and 30 d are arranged is not limited to the front and upper side in the cab 4.
- the imaging device 30c is used as a reference for the imaging devices 30a, 30b, 30c, and 30d.
- the coordinate system (Xs, Ys, Zs) of the imaging device 30c is appropriately referred to as an imaging device coordinate system.
- the origin of the imaging device coordinate system is the center of the imaging device 30c.
- the origin of each coordinate system of the imaging device 30a, the imaging device 30b, and the imaging device 30d is the center of each imaging device.
- FIG. 4 is a diagram illustrating a calibration system 50 according to the embodiment.
- the calibration system 50 includes a plurality of imaging devices 30 a, 30 b, 30 c, 30 d and the processing device 20. These are provided in the vehicle body 1 of the excavator 100 as shown in FIGS. 1 and 2.
- the plurality of imaging devices 30 a, 30 b, 30 c, and 30 d are attached to a hydraulic excavator 100 that is a work machine, images a target, and outputs an image of the target obtained by the imaging to the processing device 20.
- the processing device 20 includes a processing unit 21, a storage unit 22, and an input / output unit 23.
- the processing unit 21 is realized by a processor such as a CPU (Central Processing Unit) and a memory, for example.
- the processing device 20 implements the calibration method according to the embodiment.
- the processing unit 21 reads and executes the computer program stored in the storage unit 22. This computer program is for causing the processing unit 21 to execute the calibration method according to the embodiment.
- the processing device 20 executes the calibration method according to the embodiment, the processing device 20 performs image processing in a stereo system on at least a pair of images captured by the pair of imaging devices 30, and specifically the target position, specifically. Find the coordinates of the object in the 3D coordinate system.
- the processing device 20 can measure the target three-dimensionally using a pair of images obtained by capturing the same target with at least the pair of imaging devices 30. That is, at least a pair of the imaging device 30 and the processing device 20 measures a target three-dimensionally by a stereo method.
- At least a pair of the imaging device 30 and the processing device 20 correspond to a first position detection unit that is provided in the excavator 100 and detects a target position.
- the imaging device 30 has a function of performing three-dimensional measurement of an object by executing stereo image processing, at least a pair of the imaging devices 30 corresponds to the first position detection unit.
- the storage unit 22 is a nonvolatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), etc. At least one of a magnetic disk, a flexible disk, and a magneto-optical disk is used.
- the storage unit 22 stores a computer program for causing the processing unit 21 to execute the calibration method according to the embodiment.
- the storage unit 22 stores information used when the processing unit 21 executes the calibration method according to the embodiment.
- This information includes, for example, the orientation of each imaging device 30, the positional relationship between the imaging devices 30, the known dimensions of the work machine 2, and the known relationship indicating the positional relationship between the imaging device 30 and a fixed object mounted on the excavator 100. Necessary for obtaining the position of a part of the work implement 2 from the known dimensions indicating the positional relationship from the origin of the vehicle body coordinate system to each of the image capture devices 30 or any one of the image capture devices 30 and the orientation of the work implement 2 Information.
- the input / output unit 23 is an interface circuit for connecting the processing device 20 and devices.
- the input / output unit 23 is connected to the hub 51, the input device 52, the first angle detection unit 18A, the second angle detection unit 18B, and the third angle detection unit 18C.
- the hub 51 is connected to a plurality of imaging devices 30a, 30b, 30c, and 30d.
- the imaging device 30 and the processing device 20 may be connected without using the hub 51.
- Results obtained by the imaging devices 30 a, 30 b, 30 c, and 30 d are input to the input / output unit 23 via the hub 51.
- the processing unit 21 acquires the results imaged by the imaging devices 30a, 30b, 30c, and 30d via the hub 51 and the input / output unit 23.
- the input device 52 is used to give the input / output unit 23 information necessary when the processing unit 21 executes the calibration method according to the embodiment.
- the input device 52 is exemplified by a switch and a touch panel, but is not limited thereto.
- the input device 52 is provided in the cab 4 shown in FIG. 2, more specifically in the vicinity of the driver seat 4 ⁇ / b> S.
- the input device 52 may be attached to at least one of the right lever 25R and the left lever 25L of the operation device 25, or may be provided on the monitor panel 26 in the cab 4. Further, the input device 52 may be detachable from the input / output unit 23, or may provide information to the input / output unit 23 by wireless communication using radio waves or infrared rays.
- the processing device 20 may be realized by dedicated hardware, or a plurality of processing circuits may cooperate to realize the function of the processing device 20.
- a predetermined position of the work machine 2 in the system (Xm, Ym, Zm) is obtained.
- the predetermined position of the work machine 2 obtained from the dimensions and operating angles ⁇ 1, ⁇ 2, and ⁇ 3 of the work machine 2 is, for example, the position of the blade 9 of the bucket 8 that the work machine 2 has, the position of the bucket pin 15, and the first link pin.
- the first angle detector 18A, the second angle detector 18B, and the third angle detector 18C correspond to a position detector that detects the position of the excavator 100 that is the working machine of the embodiment, for example, the position of the work implement 2. .
- the predetermined position of the excavator 100 detected by the position detector is the same as the predetermined position of the work implement 2 that is the target of imaging of at least the pair of imaging devices 30. It is.
- the predetermined position of the excavator 100 detected by the position detector is the predetermined position of the work machine 2, but if the predetermined position of the elements constituting the excavator 100 is used, It is not limited to a predetermined position.
- a stereo camera is configured by the combination of the pair of imaging devices 30a and 30b and the combination of the pair of imaging devices 30c and 30d shown in FIG.
- the imaging devices 30a, 30b, 30c, and 30d of the excavator 100 are subjected to external calibration and vehicle body calibration before the excavator 100 is used for actual work.
- the external calibration is an operation for obtaining the position and orientation between the pair of imaging devices 30. Specifically, the external calibration obtains the position and orientation of the pair of imaging devices 30a and 30b and the position and orientation of the pair of imaging devices 30c and 30d. If these pieces of information cannot be obtained, three-dimensional measurement by the stereo method cannot be realized.
- Equation (1) The relationship between the position and orientation of the pair of imaging devices 30a and 30b is obtained by Equation (1), and the relationship between the position and orientation of the pair of imaging devices 30c and 30d is obtained by Equation (2).
- Pa is the position of the imaging device 30a
- Pb is the position of the imaging device 30b
- Pc is the position of the imaging device 30c
- Pd is the position of the imaging device 30d.
- R1 is a rotation matrix for converting the position Pb to the position Pa
- R2 is a rotation matrix for converting the position Pd to the position Pc.
- T1 is a parallel train for converting the position Pb to the position Pa
- R2 is a parallel train for converting the position Pd to the position Pc.
- Pa R1, Pb + T1, (1)
- Pc R2 / Pd + T2 (2)
- Car body calibration is an operation for obtaining the positional relationship between the imaging device 30 and the car body 1 of the excavator 100.
- Car body calibration is also called internal calibration.
- the positional relationship between the imaging device 30a and the vehicle body 1 and the positional relationship between the imaging device 30c and the vehicle body 1 are obtained. If these positional relationships cannot be obtained, the result of three-dimensional measurement by the stereo method cannot be converted to the on-site coordinate system.
- the positional relationship between the imaging device 30a and the vehicle body 1 is Equation (3)
- the positional relationship between the imaging device 30b and the vehicle body 1 is Equation (4)
- the positional relationship between the imaging device 30c and the vehicle body 1 is Equation (5)
- the positional relationship between the imaging device 30d and the vehicle body 1 is obtained by Expression (6).
- Pma is the position of the imaging device 30a in the vehicle body coordinate system
- Pmb is the position of the imaging device 30b in the vehicle body coordinate system
- Pmc is the position of the imaging device 30c in the vehicle body coordinate system
- Pmd is the position of the imaging device 30d in the vehicle body coordinate system.
- R3 is a rotation matrix for converting the position Pa into a position in the vehicle body coordinate system
- R4 is a rotation matrix for converting the position Pb into a position in the vehicle body coordinate system
- R5 is a position in the vehicle body coordinate system
- R6 is a rotation matrix for converting the position Pd into a position in the vehicle body coordinate system.
- T3 is a parallel progression for converting the position Pa into a position in the vehicle coordinate system
- T4 is a parallel progression for converting the position Pb into a position in the vehicle coordinate system
- T5 is a position in the vehicle coordinate system.
- T6 is a parallel progression for converting the position Pd into a position in the vehicle body coordinate system.
- Pma R3 ⁇ Pa + T3 (3)
- Pmb R4 ⁇ Pb + T4 ⁇ (4)
- Pmc R5 ⁇ Pc + T5 ⁇ (5)
- Pmd R6.Pd + T6 .. (6)
- the processing apparatus 20 calculates
- the rotation matrices R3, R4, R5, and R6 are the rotation angle ⁇ around the Xm axis, the rotation angle ⁇ around the Ym axis, and the rotation angle ⁇ around the Zm axis in the vehicle body coordinate system (Xm, Ym, Zm) shown in FIG. including.
- the parallel rows T3, T4, T5, and T6 include a size xm in the Xm direction, a size ym in the Ym direction, and a size zm in the Zm direction.
- the sizes xm, ym, and zm, which are elements of the parallel progression row T3, represent the position of the imaging device 30a in the vehicle body coordinate system.
- the sizes xm, ym, and zm, which are elements of the parallel progression T4 represent the position of the imaging device 30b in the vehicle body coordinate system.
- the sizes xm, ym, and zm, which are elements of the parallel progression T5 represent the position of the imaging device 30c in the vehicle body coordinate system.
- the sizes xm, ym, and zm, which are elements of the parallel progression T6 represent the position of the imaging device 30d in the vehicle body coordinate system.
- the rotation angles ⁇ , ⁇ , and ⁇ included in the rotation matrix R3 represent the attitude of the imaging device 30a in the vehicle body coordinate system.
- the rotation angles ⁇ , ⁇ , and ⁇ included in the rotation matrix R4 represent the attitude of the imaging device 30b in the vehicle body coordinate system.
- the rotation angles ⁇ , ⁇ , and ⁇ included in the rotation matrix R5 represent the attitude of the imaging device 30c in the vehicle body coordinate system.
- the rotation angles ⁇ , ⁇ , and ⁇ included in the rotation matrix R6 represent the attitude of the imaging device 30d in the vehicle body coordinate system.
- the hydraulic excavator 100 is subjected to external calibration and body calibration before being shipped from a factory, for example. These results are stored in the storage unit 22 of the processing device 20 shown in FIG.
- external calibration is performed, for example, using a measuring instrument called a total station as a yard (yagura), which is a dedicated facility, installed in the factory building, and a device for calibration.
- This kite is a large structure that is calibrated by a steel member or the like having a width of several meters and a height of nearly 10 meters.
- the calibration system 50 realizes external calibration and body calibration of the imaging device 30 at the work site of the excavator 100 by executing the calibration method according to the embodiment. Specifically, the calibration system 50 uses a predetermined position of the work machine 2, in the embodiment, the position of the blade 9 of the bucket 8, and the positions of the blades 9 of the plurality of buckets 8 obtained with different postures of the work machine 2. Both external calibration and vehicle body calibration are realized using a predetermined position outside the excavator 100. Details of the predetermined position outside the excavator 100 will be described with reference to FIG.
- FIG. 5 is a diagram illustrating objects to be imaged by the imaging device 30 when the processing device 20 according to the embodiment executes the calibration method according to the embodiment.
- the calibration system 50 uses the position of the target Tg attached to the blade 9 of the bucket 8 as a predetermined position of the work machine 2.
- the target Tg is a first sign placed at a predetermined position on the work machine 2.
- the target Tg is attached to the blades 9L, 9C, 9R, for example.
- the blade 9L is disposed at the left end
- the blade 9L is disposed at the right end
- the blade 9C is disposed at the center.
- the excavator 100 is provided with the bucket of the other form which is not provided with the blade 9, for example, is called a slope bucket. Also good.
- the target Tg is used for calibration of at least the pair of imaging devices 30, the predetermined position of the work machine 2 and the predetermined position outside the excavator 100 are reliably detected.
- the target Tg is a white background with black dots. Since the contrast becomes clear by such a target, the predetermined position of the work machine 2 and the predetermined position outside the excavator 100 are more reliably detected.
- the targets Tg are arranged in the width direction W of the bucket 8, that is, in a direction parallel to the direction in which the bucket pin 15 extends.
- the width direction W of the bucket 8 is a direction in which at least one of the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d is arranged.
- the direction in which both the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d are arranged is the same.
- the central blade 9 in the width direction W of the bucket 8 moves only on one plane in the vehicle body coordinate system, that is, on the Xm-Zm plane. Since the position of the central blade 9 is not easily affected by the posture variation in the width direction W of the bucket 8, the position accuracy is high.
- the bucket 8 is provided with the target Tg on the three blades 9, but the number of the target Tg, that is, the number of the blades 9 to be measured is not limited to three.
- the target Tg may be provided on at least one blade 9.
- the calibration method according to the embodiment has two or more targets. Tg is preferably provided at a position away from the bucket 8 in the width direction W in order to obtain high measurement accuracy.
- FIG. 6 is a diagram illustrating an example of an image IMG of the target Tg imaged by the imaging devices 30a, 30b, 30c, and 30d.
- FIG. 7 is a perspective view showing a position where the target Tg attached to the blade 9 of the bucket 8 is imaged by the imaging devices 30a, 30b, 30c, and 30d.
- FIG. 8 is a perspective view showing positions where the target Tg installed outside the excavator 100 is imaged by the imaging devices 30a, 30b, 30c, and 30d.
- the imaging devices 30a, 30b, 30c, and 30d image the target Tg of the blade 9 of the bucket 8
- the target Tgl is attached to the blade 9L.
- the target Tgc is attached to the blade 9C.
- the target Tgr is attached to the blade 9R.
- the target Tg When the pair of imaging devices 30a and 30b constituting the stereo camera images the target Tg, images IMG are obtained from the imaging device 30a and the imaging device 30b, respectively.
- images IMG are obtained from the imaging device 30c and the imaging device 30d, respectively.
- the position of the target Tg indicates the position of the blade 9 of the bucket 8, that is, a predetermined position of the work machine 2.
- Information on the position of the target Tg is first position information that is information on a predetermined position of the work implement 2 captured by at least the pair of imaging devices 30.
- the information on the position of the target Tg is information on the position in the image IMG, for example, information on the position of the pixels constituting the image IMG.
- the first position information is information obtained by the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d imaging the position of the target Tg that is the first marker from the work machine 2 in different postures.
- the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d are at eight positions A, B, C, D, E, F, G, and H, as shown in FIG.
- the target Tg is imaged.
- FIG. 7 shows the target Tg in the Xg-Yg-Zg coordinate system.
- the Xg axis is an axis parallel to the Xm axis of the vehicle body coordinate system of the excavator 100, and the front end of the swing body 3 of the excavator 100 is set to zero.
- the Yg axis is an axis parallel to the Ym axis of the vehicle body coordinate system of the excavator 100.
- the Zg axis is an axis parallel to the Zm axis of the vehicle body coordinate system of the excavator 100.
- the positions Yg0, Yg1, Yg2 of the target Tg in the Yg axis direction correspond to the positions of the blades 9L, 9C, 9R of the bucket 8 to which the target Tg is attached.
- a position Yg1 in the Yg axis direction is a central position in the width direction W of the bucket 8.
- the positions A, B, and C are Xg1 in the Xg-axis direction, and the positions in the Zg-axis direction are Zg1, Zg2, and Zg3, respectively.
- the positions D, E, and F are Xg2 in the Xg-axis direction, and the positions in the Zg-axis direction are Zg1, Zg2, and Zg3, respectively.
- the positions G and H the position in the Xg-axis direction is Xg3, and the positions in the Zg-axis direction are Zg2 and Zg3, respectively.
- the positions Xg1, Xg2, and Xg3 are further away from the swing body 3 of the excavator 100 in this order.
- the processing device 20 obtains the position of the blade 9C disposed at the center in the width direction W of the bucket 8 at each position A, B, C, D, E, F, G, H.
- the processing apparatus 20 is the position of each of the first angle detection unit 18A, the second angle detection unit 18B, and the third angle detection unit 18C at the respective positions A, B, C, D, E, F, G, and H.
- the detected values are acquired and the operating angles ⁇ 1, ⁇ 2, and ⁇ 3 are obtained.
- the processing device 20 obtains the position of the blade 9C from the obtained operating angles ⁇ 1, ⁇ 2, and ⁇ 3 and the lengths L1, L2, and L3 of the work implement 2.
- the position of the blade 9C thus obtained is the position of the excavator 100 in the vehicle body coordinate system.
- Information on the position of the blade 9C in the vehicle body coordinate system obtained at positions A, B, C, D, E, F, G, and H is the first angle detector 18A and the second angle detector 18B, which are position detectors.
- the third angle detector 18C is information obtained by detecting the position of the blade 9C, which is a predetermined position of the work implement 2 from the work implement 2 having a different posture, that is, second position information.
- a target Tg is installed at a predetermined position outside the excavator 100.
- the target Tg installed outside the excavator 100 is a second indicator.
- the target Tg is installed at a work site where the excavator 100 operates, for example.
- the target Tg is installed on the ground GD in front of the excavator 100.
- the target Tg is installed in, for example, a lattice shape in the first direction and the second direction orthogonal to the first direction.
- the target Tg is installed at positions of distances X1, X2, and X3 with respect to the front end 3T of the swing body 3 of the excavator 100.
- three targets Tg are arranged in the range of the distance Y1.
- the magnitudes of the distances X1, X2, X3, and Y1 are not limited to specific values, but it is preferable that the target Tg be arranged in the full imaging range of the imaging device 30.
- the distance X3 farthest from the revolving structure 3 is larger than the length in a state where the working machine 2 is extended to the maximum.
- the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d image the target Tg installed outside the excavator 100.
- the information on the position of the target Tg is third position information that is information on a predetermined position outside the excavator 100 and is captured by at least the pair of imaging devices 30.
- the information on the position of the target Tg is information on the position in the image captured by the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d, for example, information on the position of the pixels constituting the image.
- a plurality of targets Tg installed outside the hydraulic excavator 100 be shown in common to the respective imaging devices 30a, 30b, 30c, and 30d as much as possible.
- the target Tg is preferably installed so as to face each of the imaging devices 30a, 30b, 30c, and 30d.
- the target Tg may be attached to a pedestal installed on the ground GD. If there is an inclined surface that increases in height as it moves away from the excavator 100 at the calibration site, the target Tg may be installed on this inclined surface. Further, if there is a wall surface of a structure such as a building at the calibration site, the target Tg may be installed on this wall surface.
- the excavator 100 may be moved in front of the wall surface on which the target Tg is installed.
- the target Tg faces the imaging devices 30a, 30b, 30c, and 30d, so that the imaging devices 30a, 30b, 30c, and 30d can reliably image the target Tg.
- the number of target Tg to be installed is nine is shown, but the number of the target Tg may be at least six, and may be nine or more.
- the processing unit 21 of the processing device 20 uses the first position information, the second position information, and the third position information to relate to the positions and orientations of the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d. Ask for information.
- the processing unit 21 obtains conversion information used for converting the position of the object imaged by the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d from the first coordinate system to the second coordinate system.
- Information on the positions of the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d (hereinafter referred to as position information as appropriate) includes the sizes xm, ym, and zm included in the parallel progression rows X3, X4, X5, and X6. It is.
- Information about the postures of the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d includes rotation angles ⁇ , ⁇ , and rotation angles included in the rotation matrices R3, R4, R5, and R6.
- the conversion information is rotation matrices R3, R4, R5, and R6.
- the processing unit 21 processes the first position information, the second position information, and the third position information by using the bundle method, and obtains position information, posture information, and conversion information.
- the method for obtaining position information, posture information, and conversion information using the bundle method is the same as the aerial photogrammetry method.
- the position of the target Tg shown in FIG. 5 in the vehicle body coordinate system is Pm (Xm, Ym, Zm) or Pm.
- the position in the image IMG of the target Tg imaged by the imaging device 30 shown in FIG. 6 is Pg (i, j) or Pg.
- the position of the target Tg in the imaging apparatus coordinate system is set to Ps (Xs, Ys, Zs) or Ps.
- the position of the target Tg in the vehicle body coordinate system and the imaging device coordinate system is represented by three-dimensional coordinates, and the position of the target Tg in the image IMG is represented by two-dimensional coordinates.
- the relationship between the target position Ps in the image pickup apparatus coordinate system and the position Pm of the target Tg in the vehicle body coordinate system is expressed by Expression (7).
- R is a rotation matrix for converting the position Pm to the position Ps
- T is a parallel progression sequence for converting the position Pm to the position Ps.
- the rotation matrix R and the parallel progression T are different for each of the imaging devices 30a, 30b, 30c, and 30d.
- the relationship between the position Pg of the target Tg in the image IMG and the position Ps of the target in the imaging device coordinate system is expressed by Expression (8).
- Expression (8) is a calculation expression for converting the target position Ps in the three-dimensional image pickup apparatus coordinate system into the position Tg of the target Tg in the two-dimensional image IMG.
- Ps R ⁇ Pm + T ⁇ (7) (I ⁇ cx, j ⁇ cx)
- D (Xs, Ys) / Zs (8)
- D included in Equation (8) is a pixel ratio (mm / pixel) when the focal length is 1 mm. Further, (cx, cy) is called the image center, and indicates the position of the intersection between the optical axis of the imaging device 30 and the image IMG. D, cx, and cy are obtained by internal calibration.
- Expression (9) to Expression (11) are obtained for one target Tg imaged by one imaging device 30.
- f (Xm, i, j; R, T) 0 .. (9)
- f (Ym, i, j; R, T) 0 .. (10)
- f (Zm, i, j; R, T) 0 .. (11)
- the processing unit 21 creates Formula (11) from Formula (9) as many as the number of targets Tg imaged by the imaging devices 30a, 30b, 30c, and 30d.
- the processing unit 21 gives the value of the position Pm in the vehicle body coordinate system as a known coordinate.
- the processing unit 21 assumes that the coordinates of the remaining targets Tg attached to the blades 9 of the bucket 8, that is, the positions of the targets Tg attached to the blades 9 at both ends of the bucket 8, are unknown. It is assumed that the processing unit 21 does not know the coordinates of the position of the target Tg installed outside the excavator 100.
- the position of the target Tg attached to the central blade 9 in the width direction W of the bucket 8 corresponds to a reference point in aerial photogrammetry.
- the position of the target Tg attached to the blades 9 at both ends of the bucket 8 and the position of the target Tg installed outside the excavator 100 correspond to pass points in aerial photogrammetry.
- the processing unit 21 performs a total of 29 targets Tg captured by the pair of imaging devices 30 respectively.
- Expression (11) is generated from Expression (9).
- the processing unit 21 obtains the rotation matrix R and the parallel progression T from the obtained plurality of equations using the least square method.
- the processing unit 21 determines unknowns in the obtained plurality of equations by solving the obtained equations using, for example, the Newton-Raphson method. At this time, the processing unit 21 uses, as an initial value, for example, the results of external calibration and body calibration performed before the excavator 100 is shipped from the factory. For the target Tg whose coordinates are unknown, the processing unit 21 uses an estimated value. For example, the estimated value of the position of the target Tg attached to the blades 9 at both ends of the bucket 8 is the position of the target Tg attached to the central blade 9 in the width direction W of the bucket 8 and the dimension in the width direction W of the bucket 8. Obtained from. The estimated value of the position of the target Tg installed outside the excavator 100 can be a value measured from the origin of the vehicle body coordinate system of the excavator 100.
- the results of external calibration and body calibration performed before the excavator 100 is shipped from the factory are stored in the storage unit 22 shown in FIG.
- the estimated value of the position of the target Tg installed outside the excavator 100 is obtained in advance by an operator who performs calibration, for example, a serviceman or an operator of the excavator 100, and is stored in the storage unit 22.
- the processing unit 21 obtains, from the storage unit 22, the result of external calibration, the result of vehicle body calibration, and the estimated value of the position of the target Tg installed outside the excavator 100. Read and use as the initial value when solving the obtained equations.
- the processing unit 21 solves the obtained plural expressions.
- the processing unit 21 sets the values at that time as position information, posture information, and conversion information. Specifically, the sizes xm, ym, zm and the rotation angles ⁇ , ⁇ , ⁇ obtained for each of the imaging devices 30a, 30b, 30c, 30d when the calculation converges are determined by the imaging devices 30a, 30b, 30c and 30d position information and posture information.
- the conversion information is a rotation matrix R including rotation angles ⁇ , ⁇ , ⁇ and a parallel progression T having elements of sizes xm, ym, zm.
- FIG. 9 is a flowchart illustrating a processing example when the processing device 20 according to the embodiment executes the calibration method according to the embodiment.
- step S11 which is a detection step
- the processing unit 21 of the processing device 20 is attached to the blade 9 of the bucket 8 on the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d in a plurality of different postures of the work machine 2.
- the obtained target Tg is imaged.
- the processing unit 21 acquires detection values from the first angle detection unit 18A, the second angle detection unit 18B, and the third angle detection unit 18C when the working machine 2 is in each posture.
- the process part 21 calculates
- the processing unit 21 temporarily stores the obtained position of the blade 9 ⁇ / b> C in the storage unit 22.
- the processing unit 21 causes the pair of imaging devices 30 a and 30 b and the pair of imaging devices 30 c and 30 d to image the target Tg installed outside the excavator 100.
- the processing unit 21 obtains the position Pg of the target Tg in the image IMG captured by each of the imaging devices 30a, 30b, 30c, and 30d, and temporarily stores it in the storage unit 22.
- the processing unit 21 processes the first position information, the second position information, and the third position information by using the bundle method, and generates a plurality of formulas for obtaining the position information, the posture information, and the conversion information.
- step S12 the processing unit 21 sets an initial value.
- step S13 which is a calculation process, the processing unit 21 performs a bundle method calculation.
- step S14 the processing unit 21 performs calculation convergence determination.
- Step S15 changes the initial value when starting the calculation by the bundle method, and performs the calculation at Step S13 and the convergence determination at Step S14. Execute. If the processing unit 21 determines that the calculation has converged (step S14, Yes), the calibration ends. In this case, values when the calculation converges are set as position information, posture information, and conversion information.
- FIG. 10 is a diagram illustrating another example of the target Tg for obtaining the third position information.
- the pair of imaging devices 30a and 30b and the pair of imaging devices 30c and 30d image the target Tg attached to the blade 9 of the bucket 8 from the work machine 2 having a plurality of different postures.
- the ratio of the target Tg in the image captured by the pair of imaging devices 30 c and 30 d mounted downward using the target Tg installed outside the excavator 100 is shown. increase.
- the third position information is obtained from the target Tg installed outside the excavator 100. It is not limited to things.
- the target Tg may be disposed at a position larger than the width of the bucket 8 by the fixture 60.
- the fixture 60 includes a shaft member 62 to which the target Tg is attached, and a fixing member 61 attached to one end of the shaft member 62.
- the fixing member 61 has a magnet.
- the fixing member 61 is attached to the work machine 2 by, for example, attaching the target Tg and the shaft member 62 to the work machine 2 by being attracted to the work machine 2.
- the fixing member 61 can be attached to the work machine 2 and can be detached from the work machine 2.
- the fixing member 61 is attracted to the bucket pin 15 and the target Tg and the shaft member 62 are fixed to the work machine 2.
- the target Tg is disposed on the outer side in the width direction W of the bucket 8 than the target Tg attached to the blade 9 of the bucket 8.
- the processing unit 21 sets the target Tg attached to the work machine 2 by the fixture 60 and the target Tg attached to the blade 9 of the bucket 8 while changing the posture of the work machine 2.
- the imaging devices 30a and 30b and the pair of imaging devices 30c and 30d are caused to take images.
- a decrease in the ratio of the target Tg in the image taken by the pair of imaging devices 30c and 30d attached facing downward is suppressed. Is done.
- FIG. 11 is a diagram for explaining a place where at least a pair of imaging devices 30 is calibrated.
- the excavator 100 is installed in front of the inclined surface SP whose height decreases as the distance from the excavator 100 increases.
- Calibration of at least the pair of imaging devices 30 may be performed in a state where the excavator 100 is installed at a position where the inclined surface SP is in front of the excavator 100.
- the processing unit 21 applies the target Tg attached to the blade 9 of the bucket 8 to the pair of imaging devices 30 a and 30 b and the pair of imaging devices 30 c and 30 d by changing the posture of the work machine 2. Let's take an image. In this case, by moving the bucket 8 up and down on the inclined surface SP, the range in which the bucket 8 operates expands to a range lower than the surface on which the excavator 100 is installed. For this reason, the pair of imaging devices 30c and 30d attached facing downward is attached to the blade 9 of the bucket 8 when the bucket 8 is positioned in a range lower than the surface on which the excavator 100 is installed.
- the target Tg can be imaged. As a result, a decrease in the ratio of the target Tg in the image captured by the pair of imaging devices 30c and 30d attached facing downward is suppressed.
- FIG. 12 is a diagram illustrating an example of a tool used when the target Tg is installed outside the excavator 100.
- a portable terminal device 70 including a display unit that displays guidance on the target Tg on the screen 71 may be used as a tool for assisting the installation work when the target Tg is installed.
- the mobile terminal device 70 acquires an image captured by the pair of imaging devices 30 that are the objects of calibration from the processing device 20 of the excavator 100.
- the portable terminal device 70 displays the image which the imaging device 30 imaged on the screen 71 of a display part with the guide frames 73 and 74.
- Guide frames 73 and 74 indicate ranges that can be used for stereo matching in a pair of images captured by the pair of imaging devices 30.
- stereo matching a corresponding portion in a pair of images captured by the pair of imaging devices 30 is searched. Since the imaging ranges of the pair of imaging devices 30 are different from each other, a common portion of the range captured by the pair of imaging devices 30 is a search target, that is, a range that can be used for stereo matching (three-dimensional measurement).
- the guide frames 73 and 74 are images showing a common portion of the range captured by the pair of imaging devices 30.
- an image captured by one imaging device 30 is displayed on the left side of the screen 71, and an image captured by the other imaging device 30 is displayed on the right side of the screen 71.
- five targets Tg1, Tg2, Tg3, Tg4, and Tg5 appear in each image. All targets Tg 1, Tg 2, Tg 3, Tg 4, and Tg 5 are inside the guide frame 73, but the target Tg 1 is outside the guide frame 74. In this case, the target Tg is not used for calibration, and the accuracy of calibration cannot be ensured. For this reason, the operator who performs calibration adjusts the position of the target Tg5 so that the target Tg5 enters the guide frame 74 while visually recognizing the screen 71 of the mobile terminal device 70.
- the calibration operator can place many targets Tg in a range that can be used for stereo matching of the pair of imaging devices 30, as described above.
- the target Tg can be arranged over the entire range.
- the calibration accuracy according to the embodiment is improved. Since the image captured by the guide frames 73 and 74 and the pair of imaging devices 30 is displayed on the screen of the mobile terminal device 70, the operator who performs calibration can check the result while installing the target Tg. The work efficiency when installing Tg is improved.
- the pair of images captured by the pair of imaging devices 30 are displayed on the screen 71 of the display unit provided in the mobile terminal device 70.
- the pair of imaging devices 30a and 30b and the pair of the excavator 100 are included.
- a total of four images captured by the imaging devices 30 c and 30 d may be displayed on the screen 71.
- the images captured by the guide frames 73 and 74 and the pair of imaging devices 30 may be displayed on a screen other than the screen 71 of the mobile terminal device 70.
- the images captured by the guide frames 73 and 74 and the pair of imaging devices 30 may be displayed on the monitor panel 26 provided in the cab 4 of the excavator 100. In this way, the portable terminal device 70 becomes unnecessary.
- the predetermined position of the work implement 2 is imaged by at least the pair of imaging devices 30, and the first position information regarding the predetermined position of the work implement 2 is obtained from the obtained image.
- Second position information relating to a predetermined position at the time of imaging is obtained by a position detector different from at least the pair of imaging devices 30, and a predetermined position outside the work machine is imaged by at least the pair of imaging devices 30,
- Third position information relating to a predetermined position outside the work machine is obtained from the obtained image.
- the calibration system 50 and the calibration method according to the embodiment use the first position information, the second position information, and the third position information, and at least information about the position and orientation of the pair of imaging devices 30; Conversion information used to convert the position of the object imaged by the pair of imaging devices 30 from the first coordinate system to the second coordinate system is obtained.
- the calibration system 50 and the calibration method according to the embodiment can simultaneously perform external calibration and vehicle body calibration of at least a pair of imaging devices 30 attached to the work machine.
- information necessary for calibration is obtained by imaging a predetermined position of the work machine 2 and a predetermined position outside the work machine with at least a pair of imaging devices 30. Therefore, at least a pair of imaging devices 30 can be calibrated even in a work site where it is difficult to prepare calibration equipment, personnel for operating the equipment, dedicated equipment, and the like.
- the target Tg since the target Tg is installed outside the work machine in addition to the target Tg attached to the work machine 2, a wide image captured by at least the pair of imaging devices 30 is obtained.
- the target Tg can be present in the range.
- the target Tg since the target Tg is installed outside the work machine, a decrease in the ratio of the target Tg in the image captured by the pair of imaging devices 30c and 30s installed facing downward is suppressed. As a result, the ground can be reliably three-dimensionally measured by the stereo method, and the measurement accuracy can be improved.
- the second position information may be a plurality of pieces of information obtained from at least three different postures of the work implement 2.
- two pairs of imaging devices 30 are calibrated.
- the calibration system 50 and the calibration method according to the embodiment can also be applied to calibration of a pair of imaging devices 30 and calibration of three or more pairs of imaging devices 30. .
- the position detector is the first angle detector 18A, the second angle detector 18B, and the third angle detector 18C, but is not limited thereto.
- the excavator 100 is equipped with an antenna for RTK-GNSS (Real Time Kinematic-Global Navigation Satellite Systems, GNSS is a global navigation satellite system), and the position of the vehicle is measured by measuring the position of the antenna by GNSS. It is assumed that a position detection system for detection is provided. In this case, the position detection system described above is used as a position detector, and the position of the GNSS antenna is set as a predetermined position of the work machine.
- RTK-GNSS Real Time Kinematic-Global Navigation Satellite Systems
- the first position information and the second position information are obtained by detecting the position of the GNSS antenna with at least a pair of the imaging device 30 and the position detector while changing the position of the GNSS antenna.
- the processing unit 21 uses the first position information and the second position information obtained and the third position information obtained from the target Tg installed outside the work machine to obtain position information, posture information, and conversion information. Ask for.
- a removable GNSS receiver is attached to a predetermined position of the excavator 100, for example, a predetermined position of the traveling body 5 or the work machine 2, and the GNSS receiver is used as a position detector as described above. Conversion information is obtained in the same manner as when the position detection system for detecting the position of the vehicle is a position detector.
- the work machine is not limited to the hydraulic excavator 100 as long as it includes at least a pair of imaging devices 30 and three-dimensionally measures an object in a stereo manner using at least the pair of imaging devices 30.
- the work machine may have a work machine, and may be a work machine such as a wheel loader or a bulldozer.
- the target Tg is provided on the blade 9, but these are not necessarily required.
- the input device 52 shown in FIG. 4 specifies a portion for which the processing unit 21 obtains a position, for example, the portion of the blade 9 of the bucket 8, in the target image captured by at least the pair of imaging devices 30. Good.
Landscapes
- Engineering & Computer Science (AREA)
- Mining & Mineral Resources (AREA)
- Civil Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Structural Engineering (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Component Parts Of Construction Machinery (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Operation Control Of Excavators (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
Description
図1は、実施形態に係る校正システムを備えた油圧ショベル100の斜視図である。図2は、実施形態に係る油圧ショベル100の運転席付近を斜視図である。図3は、実施形態に係る油圧ショベルが有する作業機2の寸法及び油圧ショベル100の座標系を示す図である。
図2に示されるように、油圧ショベル100は、例えば、運転室4内に、複数の撮像装置30a,30b,30c,30dを有する。以下において、複数の撮像装置30a,30b,30c,30dを区別しない場合は適宜、撮像装置30と称する。撮像装置30の種類は限定されないが、実施形態では、例えば、CCD(Couple Charged Device)イメージセンサ又はCMOS(Complementary Metal Oxide Semiconductor)イメージセンサを備えた撮像装置が用いられる。
図4は、実施形態に係る校正システム50を示す図である。校正システム50は、複数の撮像装置30a,30b,30c,30dと、処理装置20とを含む。これらは、図1及び図2に示されるように、油圧ショベル100の車体1に備えられている。複数の撮像装置30a,30b,30c,30dは、作業機械である油圧ショベル100に取り付けられて、対象を撮像し、撮像によって得られた対象の画像を処理装置20に出力する。
実施形態において、図2に示される一対の撮像装置30a,30bの組合せと、一対の撮像装置30c,30dの組合せとで、それぞれステレオカメラが構成される。油圧ショベル100が有する撮像装置30a,30b,30c,30dは、油圧ショベル100が実際の作業に使用される前に、外部校正及び車体校正が行われる。外部校正は、一対の撮像装置30同士の位置及び姿勢を求める作業である。詳細には、外部校正は、一対の撮像装置30a,30b同士の位置及び姿勢と、一対の撮像装置30c,30d同士の位置及び姿勢とを求める。これらの情報が得られないと、ステレオ方式による三次元計測は実現できない。
Pa=R1・Pb+T1・・(1)
Pc=R2・Pd+T2・・(2)
Pma=R3・Pa+T3・・(3)
Pmb=R4・Pb+T4・・(4)
Pmc=R5・Pc+T5・・(5)
Pmd=R6・Pd+T6・・(6)
Ps=R・Pm+T・・(7)
(i-cx,j-cx)D=(Xs,Ys)/Zs・・(8)
f(Xm,i,j;R,T)=0・・(9)
f(Ym,i,j;R,T)=0・・(10)
f(Zm,i,j;R,T)=0・・(11)
図10は、第3位置情報を得るためのターゲットTgの他の例を示す図である。前述したように、一対の撮像装置30a,30b及び一対の撮像装置30c,30dは、異なる複数の姿勢の作業機2から、バケット8の刃9に取り付けられたターゲットTgを撮像する。図10に示される例では、油圧ショベル100の外部に設置されたターゲットTgを用いて、下方を向いて取り付けられた一対の撮像装置30c,30dによって撮像された画像中に占めるターゲットTgの割合を増加させる。
図12は、油圧ショベル100の外部にターゲットTgを設置する際に用いられるツールの一例を示す図である。ターゲットTgの設置時に、設置作業の補助するツールとして、例えば、ターゲットTgのガイダンスを画面71に表示する表示部を備えた携帯端末装置70が用いられてもよい。この例において、携帯端末装置70は、校正の対象となっている一対の撮像装置30が撮像している画像を、油圧ショベル100の処理装置20から取得する。そして、携帯端末装置70は、撮像装置30が撮像している画像を、ガイド枠73,74とともに表示部の画面71に表示する。
2 作業機
3 旋回体
3T 前端
4 運転室
5 走行体
6 ブーム
7 アーム
8 バケット
9,9L,9C,9R 刃
10 ブームシリンダ
11 アームシリンダ
12 バケットシリンダ
18A 第1角度検出部
18B 第2角度検出部
18C 第3角度検出部
20 処理装置
21 記憶部
21 処理部
22 記憶部
23 入出力部
30,30a,30b,30c,30d 撮像装置
50 校正システム
100 油圧ショベル
Tg,Tg1,Tg2,Tg3,Tg4,Tg5,Tgl,Tgc,Tgr ターゲット
Claims (6)
- 作業機を有する作業機械に備えられて、対象を撮像する少なくとも一対の撮像装置と、
前記作業機の位置を検出する位置検出器と、
少なくとも一対の前記撮像装置によって撮像された前記作業機の所定の位置に関する情報である第1位置情報と、少なくとも一対の前記撮像装置が前記所定の位置を撮像したときの前記作業機の姿勢で、前記位置検出器によって検出された前記所定の位置に関する情報である第2位置情報と、少なくとも一対の前記撮像装置によって撮像された前記作業機械の外部における所定の位置に関する情報である第3位置情報と、を用いて、
少なくとも一対の前記撮像装置の位置及び姿勢に関する情報と、少なくとも一対の前記撮像装置が撮像した前記対象の位置を第1の座標系から第2の座標系に変換するために用いられる変換情報と、を求める処理部と、
を含む、校正システム。 - 前記作業機の前記所定の位置には第1標識が配置されており、前記第1位置情報は、少なくとも一対の前記撮像装置が、異なる姿勢の前記作業機から前記第1標識の位置を撮像することにより得られた情報であり、
前記第2位置情報は、前記位置検出器が、異なる姿勢の前記作業機から前記所定の位置を検出することにより得られた情報であり、
前記第3位置情報は、前記作業機械の外部に設置された第2標識の位置の情報である、請求項1に記載の校正システム。 - 前記第2位置情報は、少なくとも一対の前記撮像装置が配列されている方向における、前記作業機の中央の位置に関する情報であり、前記作業機の異なる少なくとも3つの姿勢で得られた複数の情報である、請求項1又は請求項2に記載の校正システム。
- 前記位置検出器は、前記作業機械に備えられて、前記作業機を動作させるアクチュエータの動作量を検出するセンサである、請求項1又は請求項2に記載の校正システム。
- 前記作業機と、
請求項1から請求項4のいずれか1項に記載の前記校正システムと、
を含む、作業機械。 - 少なくとも一対の撮像装置によって作業機の所定の位置及び前記作業機を有する作業機械の周囲の所定の位置を撮像するとともに、少なくとも一対の前記撮像装置とは異なる位置検出器によって前記作業機械の所定の位置を検出する検出工程と、
少なくとも一対の前記撮像装置によって撮像された前記作業機の所定の位置に関する情報である第1位置情報と、少なくとも一対の前記撮像装置が前記所定の位置を撮像したときの前記作業機の姿勢で、前記位置検出器によって検出された前記所定の位置に関する情報である第2位置情報と、少なくとも一対の前記撮像装置によって撮像された前記作業機械の外部における所定の位置に関する情報である第3位置情報と、を用いて、少なくとも一対の前記撮像装置の位置及び姿勢に関する情報と、少なくとも一対の前記撮像装置が撮像した対象の位置を第1の座標系から第2の座標系に変換するために用いられる変換情報と、を求める演算工程と、
を含む、校正方法。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE112016000038.3T DE112016000038B4 (de) | 2016-03-29 | 2016-03-29 | Kalibrierungssystem, Arbeitsmaschine und Kalibrierungsverfahren |
CN201680000572.7A CN106029994B (zh) | 2016-03-29 | 2016-03-29 | 校正系统、作业机械和校正方法 |
JP2017506234A JP6229097B2 (ja) | 2016-03-29 | 2016-03-29 | 校正システム、作業機械及び校正方法 |
US15/308,453 US20170284071A1 (en) | 2016-03-29 | 2016-03-29 | Calibration system, work machine, and calibration method |
PCT/JP2016/060273 WO2016148309A1 (ja) | 2016-03-29 | 2016-03-29 | 校正システム、作業機械及び校正方法 |
KR1020167029420A KR101885704B1 (ko) | 2016-03-29 | 2016-03-29 | 교정 시스템, 작업 기계 및 교정 방법 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2016/060273 WO2016148309A1 (ja) | 2016-03-29 | 2016-03-29 | 校正システム、作業機械及び校正方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016148309A1 true WO2016148309A1 (ja) | 2016-09-22 |
Family
ID=56919811
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2016/060273 WO2016148309A1 (ja) | 2016-03-29 | 2016-03-29 | 校正システム、作業機械及び校正方法 |
Country Status (6)
Country | Link |
---|---|
US (1) | US20170284071A1 (ja) |
JP (1) | JP6229097B2 (ja) |
KR (1) | KR101885704B1 (ja) |
CN (1) | CN106029994B (ja) |
DE (1) | DE112016000038B4 (ja) |
WO (1) | WO2016148309A1 (ja) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018079789A1 (ja) * | 2016-10-31 | 2018-05-03 | 株式会社小松製作所 | 計測システム、作業機械及び計測方法 |
JP2018184815A (ja) * | 2017-04-27 | 2018-11-22 | 株式会社小松製作所 | 撮像装置の校正装置、作業機械および校正方法 |
JP2019214835A (ja) * | 2018-06-11 | 2019-12-19 | 株式会社小松製作所 | 作業機械を含むシステム、コンピュータによって実行される方法、学習済みの位置推定モデルの製造方法、および学習用データ |
WO2020003497A1 (ja) * | 2018-06-29 | 2020-01-02 | 株式会社小松製作所 | 撮像装置の校正装置、監視装置、作業機械および校正方法 |
WO2020059220A1 (ja) * | 2018-09-21 | 2020-03-26 | 日立建機株式会社 | 座標変換システム及び作業機械 |
EP3561182A4 (en) * | 2016-12-22 | 2020-07-15 | Kubota Corporation | CONSTRUCTION MACHINE |
JP2021183448A (ja) * | 2020-05-22 | 2021-12-02 | 鉄建建設株式会社 | 建設車両用映像表示システム |
KR20230006651A (ko) | 2020-06-19 | 2023-01-10 | 가부시키가이샤 고마쓰 세이사쿠쇼 | 교정 장치 및 교정 방법 |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018017617A (ja) * | 2016-07-28 | 2018-02-01 | 株式会社神戸製鋼所 | 建設機械 |
DE102017114450B4 (de) * | 2017-06-29 | 2020-10-08 | Grammer Aktiengesellschaft | Vorrichtung und Verfahren zum Abbilden von Bereichen |
US10422111B2 (en) * | 2017-07-13 | 2019-09-24 | Komatsu Ltd. | Hydraulic excavator and hydraulic excavator calibration method |
US10526766B2 (en) * | 2017-07-31 | 2020-01-07 | Deere & Company | Work machines and methods and systems to control and determine a position of an associated implement |
US11195351B2 (en) | 2017-09-01 | 2021-12-07 | Komatsu Ltd. | Work machine measurement system, work machine, and measuring method for work machine |
JP6840645B2 (ja) * | 2017-09-08 | 2021-03-10 | 株式会社小松製作所 | 施工管理装置および施工管理方法 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001055762A (ja) * | 1999-08-13 | 2001-02-27 | Hitachi Constr Mach Co Ltd | 自動運転建設機械およびその位置計測手段の校正方法 |
JP2012233353A (ja) * | 2011-05-02 | 2012-11-29 | Komatsu Ltd | 油圧ショベルの較正システム及び油圧ショベルの較正方法 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5227139B2 (ja) * | 2008-11-12 | 2013-07-03 | 株式会社トプコン | 建設機械 |
US9139977B2 (en) * | 2010-01-12 | 2015-09-22 | Topcon Positioning Systems, Inc. | System and method for orienting an implement on a vehicle |
US8965642B2 (en) * | 2012-10-05 | 2015-02-24 | Komatsu Ltd. | Display system of excavating machine and excavating machine |
JP6258582B2 (ja) * | 2012-12-28 | 2018-01-10 | 株式会社小松製作所 | 建設機械の表示システムおよびその制御方法 |
JP6232494B2 (ja) * | 2014-04-23 | 2017-11-15 | 株式会社日立製作所 | 掘削装置 |
US9666843B2 (en) * | 2014-07-30 | 2017-05-30 | Ford Global Technologies, Llc | Array frame design for electrified vehicle battery arrays |
US9824490B1 (en) * | 2015-06-08 | 2017-11-21 | Bentley Systems, Incorporated | Augmentation of a dynamic terrain surface |
-
2016
- 2016-03-29 WO PCT/JP2016/060273 patent/WO2016148309A1/ja active Application Filing
- 2016-03-29 DE DE112016000038.3T patent/DE112016000038B4/de not_active Expired - Fee Related
- 2016-03-29 US US15/308,453 patent/US20170284071A1/en not_active Abandoned
- 2016-03-29 JP JP2017506234A patent/JP6229097B2/ja active Active
- 2016-03-29 CN CN201680000572.7A patent/CN106029994B/zh not_active Expired - Fee Related
- 2016-03-29 KR KR1020167029420A patent/KR101885704B1/ko active IP Right Grant
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001055762A (ja) * | 1999-08-13 | 2001-02-27 | Hitachi Constr Mach Co Ltd | 自動運転建設機械およびその位置計測手段の校正方法 |
JP2012233353A (ja) * | 2011-05-02 | 2012-11-29 | Komatsu Ltd | 油圧ショベルの較正システム及び油圧ショベルの較正方法 |
Cited By (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6995767B2 (ja) | 2016-10-31 | 2022-01-17 | 株式会社小松製作所 | 計測システム、作業機械及び計測方法 |
JPWO2018079789A1 (ja) * | 2016-10-31 | 2019-09-19 | 株式会社小松製作所 | 計測システム、作業機械及び計測方法 |
WO2018079789A1 (ja) * | 2016-10-31 | 2018-05-03 | 株式会社小松製作所 | 計測システム、作業機械及び計測方法 |
US11441294B2 (en) | 2016-10-31 | 2022-09-13 | Komatsu Ltd. | Measurement system, work machine, and measurement method |
EP3561182A4 (en) * | 2016-12-22 | 2020-07-15 | Kubota Corporation | CONSTRUCTION MACHINE |
US11408148B2 (en) | 2016-12-22 | 2022-08-09 | Kubota Corporation | Working machine with control device to control operation allowable state and operation restriction state |
JP2018184815A (ja) * | 2017-04-27 | 2018-11-22 | 株式会社小松製作所 | 撮像装置の校正装置、作業機械および校正方法 |
JP2019214835A (ja) * | 2018-06-11 | 2019-12-19 | 株式会社小松製作所 | 作業機械を含むシステム、コンピュータによって実行される方法、学習済みの位置推定モデルの製造方法、および学習用データ |
WO2019239668A1 (ja) * | 2018-06-11 | 2019-12-19 | 株式会社小松製作所 | 作業機械を含むシステム、コンピュータによって実行される方法、学習済みの位置推定モデルの製造方法、および学習用データ |
US11814817B2 (en) | 2018-06-11 | 2023-11-14 | Komatsu Ltd. | System including work machine, computer implemented method, method for producing trained position estimation model, and training data |
JP7177608B2 (ja) | 2018-06-11 | 2022-11-24 | 株式会社小松製作所 | 作業機械を含むシステム、コンピュータによって実行される方法、学習済みの位置推定モデルの製造方法、および学習用データ |
JP7203105B2 (ja) | 2018-06-29 | 2023-01-12 | 株式会社小松製作所 | 撮像装置の校正装置、監視装置、作業機械および校正方法 |
JPWO2020003497A1 (ja) * | 2018-06-29 | 2021-07-15 | 株式会社小松製作所 | 撮像装置の校正装置、監視装置、作業機械および校正方法 |
CN112334733A (zh) * | 2018-06-29 | 2021-02-05 | 株式会社小松制作所 | 拍摄装置的校正装置、监视装置、作业机械及校正方法 |
US11508091B2 (en) | 2018-06-29 | 2022-11-22 | Komatsu Ltd. | Calibration device for imaging device, monitoring device, work machine and calibration method |
WO2020003497A1 (ja) * | 2018-06-29 | 2020-01-02 | 株式会社小松製作所 | 撮像装置の校正装置、監視装置、作業機械および校正方法 |
JP2020051029A (ja) * | 2018-09-21 | 2020-04-02 | 日立建機株式会社 | 座標変換システム及び作業機械 |
WO2020059220A1 (ja) * | 2018-09-21 | 2020-03-26 | 日立建機株式会社 | 座標変換システム及び作業機械 |
JP7301514B2 (ja) | 2018-09-21 | 2023-07-03 | 日立建機株式会社 | 座標変換システム及び作業機械 |
US11906641B2 (en) | 2018-09-21 | 2024-02-20 | Hitachi Construction Machinery Co., Ltd. | Work machine |
JP2021183448A (ja) * | 2020-05-22 | 2021-12-02 | 鉄建建設株式会社 | 建設車両用映像表示システム |
JP7428588B2 (ja) | 2020-05-22 | 2024-02-06 | 鉄建建設株式会社 | 建設車両用映像表示システム |
KR20230006651A (ko) | 2020-06-19 | 2023-01-10 | 가부시키가이샤 고마쓰 세이사쿠쇼 | 교정 장치 및 교정 방법 |
DE112021002347T5 (de) | 2020-06-19 | 2023-01-26 | Komatsu Ltd. | Kalibriervorrichtung und Kalibrierverfahren |
Also Published As
Publication number | Publication date |
---|---|
JP6229097B2 (ja) | 2017-11-08 |
KR20170112999A (ko) | 2017-10-12 |
DE112016000038T5 (de) | 2017-03-23 |
DE112016000038B4 (de) | 2020-10-01 |
CN106029994A (zh) | 2016-10-12 |
KR101885704B1 (ko) | 2018-08-06 |
US20170284071A1 (en) | 2017-10-05 |
JPWO2016148309A1 (ja) | 2017-05-25 |
CN106029994B (zh) | 2020-04-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6229097B2 (ja) | 校正システム、作業機械及び校正方法 | |
JP6050525B2 (ja) | 位置計測システム、作業機械及び位置計測方法 | |
WO2016047807A1 (ja) | 校正システム、作業機械及び校正方法 | |
JP5328830B2 (ja) | 油圧ショベルの較正装置及び油圧ショベルの較正方法 | |
JP5237408B2 (ja) | 油圧ショベルの較正システム及び較正方法 | |
JP5237409B2 (ja) | 油圧ショベルの較正装置及び油圧ショベルの較正方法 | |
US11120577B2 (en) | Position measurement system, work machine, and position measurement method | |
JP6966218B2 (ja) | 撮像装置の校正装置、作業機械および校正方法 | |
US11441294B2 (en) | Measurement system, work machine, and measurement method | |
JP7203105B2 (ja) | 撮像装置の校正装置、監視装置、作業機械および校正方法 | |
WO2011040239A1 (ja) | 広角撮像装置及び測定システム | |
CN102798456B (zh) | 一种工程机械臂架系统工作幅度的测量方法、装置及系统 | |
WO2020059220A1 (ja) | 座標変換システム及び作業機械 | |
JP7023813B2 (ja) | 作業機械 | |
WO2020065738A1 (ja) | 作業機の外形形状測定システム,作業機の外形形状表示システム,作業機の制御システム及び作業機械 | |
JP5686878B2 (ja) | テールクリアランスを特定するための装置、方法、プログラムおよび記録媒体 | |
JP2017193958A (ja) | 校正システム、作業機械及び校正方法 | |
JP6616149B2 (ja) | 施工方法、作業機械の制御システム及び作業機械 | |
JP6598552B2 (ja) | 位置計測システム | |
JP2009042175A (ja) | 施工位置測定システム及び丁張りレスシステム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 20167029420 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2017506234 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16765136 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15308453 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 112016000038 Country of ref document: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16765136 Country of ref document: EP Kind code of ref document: A1 |