TW201937920A - Coordinate system integration method, and device provided with columnar body - Google Patents

Abstract

This coordinate system integration method for mapping a sensor coordinate system of an object detecting sensor to a predetermined reference coordinate system includes: a measuring step of using a second measuring device to measure a calibration marker which is mounted on an attachment jig attaching the object detecting sensor to an arm and which defines a sensor coordinate system, and a reference marker which is mounted on the arm and which defines a reference coordinate system; and an integrating step of mapping the sensor coordinate system defined by a calibration marker unit to the reference coordinate system defined by the reference marker, on the basis of the results of the measurements in the measuring step.

Description

Coordinate system integration method and device having column body

The present invention relates to a coordinate system integration method and a device having a columnar body.

For example, the industry is attached to an object detecting sensor such as a construction machine, and an object detecting sensor is used to detect an object around a construction machine or the like. When the object detecting sensor detects the object, the preset coordinate coordinate system of the construction machine or the like is associated with the set position and posture of the object detecting sensor, that is, the reference coordinate system and the object detecting sensor are The sensor coordinates are mapped.

For example, in the surveillance camera that monitors a vehicle traveling on a road, the road plane is associated with the installation position and posture of the surveillance camera, as described in the patent document (Japanese Patent No. 3,897,984).

In the method described in the above patent document, the feature point is detected by the monitoring camera, and the road plane is associated with the installation position and posture of the monitoring camera based on the detection result of the feature point. In the same manner as described above, the reference coordinate system is associated with the installation position and posture of the object detection sensor based on the detection result of the object detection sensor attached to the construction machine or the like.

However, when using the detection result of the object detecting sensor which is the object to be set corresponding to the set position and posture, the accuracy of the correspondence between the reference coordinate system and the sensor coordinate system depends on the object detecting sensor. Detection accuracy.

In view of the above, the present invention describes a coordinate system integrating method for associating a reference coordinate system of a target device with a sensor coordinate system of an object detecting sensor without depending on the detection accuracy of the object detecting sensor, and having a columnar shape Body device.

One aspect of the present invention is a coordinate system integration method that associates a sensor coordinate system of an object detection sensor mounted on an object device and detects an object around the object device with a reference coordinate system preset for the target device And including: a measuring step of setting, by the measuring device, three or more calibration markers provided on the mounting fixture for mounting the object detecting sensor to the target device and defining the sensor coordinate system; The target device defines three or more reference markers of the reference coordinate system for measurement; and an integration step based on the measurement result of the measurement step, the sensor coordinate system defined by the calibration marker and the reference marker The defined base coordinates are mapped.

In the coordinate system integration method, the measurement step is performed, and the calibration marker and the reference marker are measured by the measuring device. And, in the integration step, based on the measurement result of the measurement step, the sensor coordinate system is associated with the reference coordinate system. That is, the sensor coordinate system and the reference coordinate system are specified based on the measurement result of the measuring device, and the sensor coordinate system is associated with the reference coordinate system, and the detection result of the object detection sensor is not used. Therefore, in the coordinate system integration method, the reference coordinate system of the target device can be associated with the sensor coordinate system of the object detection sensor without depending on the detection accuracy of the object detection sensor.

The object detecting sensor may include a first object detecting sensor and a second object detecting sensor, and the mounting jig includes a first mounting jig for attaching the first object detecting sensor to the target device, And attaching the second object detecting sensor to the second mounting jig of the target device; the calibration tag includes a first sensor coordinate system that is disposed on the first mounting jig and defines the first object detecting sensor Three or more first calibration markers and three or more second calibration markers provided in the second mounting fixture and defining the second sensor coordinate system of the second object detection sensor; When three or more objects are installed in the target device, and the measurement device measures three or more first calibration markers, at least three or more of the reference markers are measured in the direction in which they are measured together with the first calibration marker. When the device measures three or more second calibration markers, at least three or more of the reference markers are oriented in the direction of being measured together with the second calibration marker; in the measurement step, three pairs of measurement devices are used. The above first calibration marker and three or more The reference marker is measured, and three or more second calibration markers and three or more reference markers are measured by the measuring device; and in the integration step, the first calibration marker is defined 1 The sensor coordinate system is associated with the reference coordinate system, and the second sensor coordinate system defined by the second calibration mark is associated with the reference coordinate system. In this case, even if the first object detecting sensor and the second object detecting sensor are disposed at different positions, the first calibration mark and the reference mark can be measured in the measuring step, and 2 Calibration is performed with the marker and the reference marker. Moreover, even if a plurality of object detecting sensors are provided, the sensor coordinate system of each object detecting sensor can be associated with the reference coordinate system.

Another aspect of the present invention is a coordinate system integration method for establishing a sensor coordinate system of an object detecting sensor mounted on an object device and detecting an object around the object device, and establishing a reference coordinate system preset for the target device Correspondingly, and including: a first measuring step, by using the first measuring device, the mounting jig or the object detecting sensor provided on the object detecting sensor to the target device and defining the sensor coordinate system The measuring unit coordinate specifying unit and the three or more calibration marks provided in the mounting jig and defining the calibration mark coordinate system are measured; and the first corresponding establishing step is based on the measurement result of the first measuring step, The sensor coordinate system defined by the sensor coordinate specific portion is associated with the calibration marker coordinate system defined by the calibration marker; and the second measurement step is performed by the second measurement device for three or more The calibration marker and the three or more reference markers provided in the target device and defining the reference coordinate system are measured; the second correspondence establishing step is based on the measurement of the second measurement step Correspondingly, the calibration marker coordinate system defined by the calibration marker is associated with the reference coordinate system defined by the reference marker; and the integration step is based on the corresponding sensor established in the first correspondence establishment step The coordinate system and the calibration marker coordinate system and the corresponding calibration marker coordinate system and the reference coordinate system are established in the second correspondence establishing step, and the sensor coordinate system is associated with the reference coordinate system.

In the coordinate system integration method, the sensor measurement target portion and the calibration mark are measured by the first measurement device in the first measurement step, and the second measurement device is used for calibration in the second measurement step. The marker and the reference marker were measured. In the first correspondence establishing step, the sensor coordinate system is associated with the calibration marker coordinate system, and in the second correspondence establishing step, the calibration marker coordinate system is associated with the reference coordinate system. And, in the integration step, the sensor coordinate system is associated with the reference coordinate system. In other words, the correspondence between the coordinate systems is performed based on the measurement results of the first measurement device and the second measurement device, and the detection result of the object detection sensor is not used. Therefore, in the coordinate system integration method, the reference coordinate system of the target device can be associated with the sensor coordinate system of the object detection sensor without depending on the detection accuracy of the object detection sensor.

The object detecting sensor may include a first object detecting sensor and a second object detecting sensor, and the mounting jig includes a first mounting jig for attaching the first object detecting sensor to the target device, And attaching the second object detecting sensor to the second mounting jig of the target device; the sensor coordinate specifying portion includes the first mounting jig or the first object detecting sensor and defining the first object detecting and sensing The first sensor coordinate specific portion of the first sensor coordinate system and the second sensor that is provided in the second mounting jig or the second object detecting sensor and defining the second object detecting sensor The second sensor coordinate specific portion of the coordinate system; the calibration marker includes three or more first calibration markers that are provided in the first mounting jig and define the first calibration marker coordinate system, and are provided in the first (2) Fixing the fixture and defining three or more second calibration markers of the second calibration marker coordinate system; the reference marker is provided in three or more target devices, and the second measurement device is configured on three or more first calibration schools. When measuring with a quasi-marker, at least three or more of the fiducial markers are oriented and aligned for the first calibration. When the second measuring device measures three or more second calibration markers, at least three or more of the reference markers are oriented in the direction of being measured together with the second calibration marker; In the first measurement step, the first sensor-specific portion and the three or more first calibration markers are measured by the first measuring device, and the second sensor coordinates are specified by the first measuring device. And the third or more calibration markers are measured; in the first correspondence establishing step, the first sensor coordinate system defined by the first sensor coordinate specifying unit and the first calibration marker are The first calibration marker coordinate system is defined, and the second sensor coordinate system defined by the second sensor coordinate specifying unit and the second calibration mark defined by the second calibration mark are used. The object coordinate system is associated with each other; in the second measurement step, three or more first calibration markers and three or more reference markers are measured by the second measuring device, and the second measuring device pair is used. More than two second calibration markers and three or more benchmarks Recording is performed; in the second correspondence establishing step, the first calibration marker coordinate system defined by the first calibration marker is associated with the reference coordinate system defined by the reference marker, and the second calibration is performed. The second calibration marker coordinate system defined by the quasi-marker is associated with the reference coordinate system defined by the fiducial marker; in the integration step, the corresponding first sensor coordinate is established based on the first correspondence establishment step And the first calibration marker coordinate system and the first calibration marker coordinate system and the reference coordinate system are established in the second correspondence establishing step, and the first sensor coordinate system is associated with the reference coordinate system. And based on the second sensor coordinate system and the second calibration marker coordinate system established in the first correspondence establishing step, and the second calibration marker coordinate system and the reference established in the second correspondence establishing step. The coordinate system associates the second sensor coordinate system with the reference coordinate system. In this case, even if the first object detecting sensor and the second object detecting sensor are disposed at different positions, the first calibration marker and the reference marker can be measured in the second measurement step, and The second calibration marker and the reference marker were measured. Moreover, even if a plurality of object detecting sensors are provided, the sensor coordinate system of each object detecting sensor can be associated with the reference coordinate system.

The object detecting sensor may also detect the object by illuminating a plurality of beams of laser light around the object device. For example, in consideration of the case where the reference detecting object is detected by the object detecting sensor, the reference mark is not detected due to the difference in the irradiation interval of the laser light. However, in the coordinate system integration method, since the detection result of the object detection sensor can be used, the sensor coordinate system is associated with the reference coordinate system, so that the detection accuracy of the object detection sensor can be performed without depending on the detection accuracy of the object detection sensor. The correspondence of the coordinate system is established.

A device having a columnar body according to still another aspect of the present invention includes: a movable columnar body; three or more reference markers, which are disposed on the columnar body and define a reference coordinate set to the columnar body An object detecting sensor that moves along with the columnar body; and three or more calibration markers, which are disposed on the object detecting sensor or the mounting fixture mounted on the object detecting sensor, And defining a sensor coordinate system of the object detection sensor; and more than three reference markers and three or more calibration markers are set to be visible from one location.

In the apparatus having the columnar body, three or more reference markers and three or more calibration markers can be visually recognized from one location. Thereby, by measuring the calibration marker and the reference marker, the sensor coordinate system can be associated with the reference coordinate system. That is, the sensor coordinate system can be associated with the reference coordinate system without using the detection result of the object detection sensor. Therefore, in the apparatus having the columnar body, the reference coordinate system set to the columnar body can be associated with the sensor coordinate system of the object detecting sensor without depending on the detection accuracy of the object detecting sensor.

According to various aspects of the present invention, the reference coordinate system of the object device can be associated with the sensor coordinate system of the object detecting sensor without depending on the detection accuracy of the object detecting sensor.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same components are denoted by the same reference numerals, and the description thereof will not be repeated.

In the present embodiment, the reference coordinate system in which the hydraulic excavator (target device, device) 1 shown in FIG. 1 is set in advance and the object detection sensors 110A to 110D attached to the hydraulic excavator 1 are referred to (refer to The sensor coordinate system of Fig. 3(a) to Fig. 3(d)) is described by establishing a corresponding coordinate system integration method. Here, the two coordinate systems are associated with each other, and the position and posture of the other coordinate system relative to one of the standard systems are grasped. That is, the two coordinate systems are associated with each other, so that coordinate conversion parameters based on translation and rotation from one standard to another coordinate system can be obtained, and the coordinates obtained by one calibration system can be converted into another coordinate system. The state of the coordinates.

The hydraulic excavator 1 is provided with: a moving body 11 for moving itself; a rotating body 10 provided on the moving body 11; a lifting arm 12 attached to the front of the rotating body 10; and an arm (columnar) 13 is mounted on the front end of the boom 12; the bucket 14 is mounted on the front end of the arm 13. By the oil pressure, the boom 12 is swung relative to the swing body 10, the arm 13 is swung relative to the boom 12, and the bucket 14 is swung relative to the arm 13.

As shown in FIGS. 1 and 2, the arms 13 are formed in a quadrangular prism shape. In the present embodiment, the axis of the arm 13 is the first axis, the axis orthogonal to the first axis is the second axis, and the axis orthogonal to the first axis and the second axis is the third axis. The coordinate system defined by the first axis, the second axis, and the third axis is used as a reference coordinate system set in advance for the hydraulic excavator 1 (arm 13). The origin of the reference coordinate system is set to a specific position in the axial direction of the arm 13.

Sensor units 100A to 100D are attached to respective surfaces of the outer peripheral surface of the arm 13 respectively. As shown in FIG. 3(a), a reference mark portion 200A is provided on the surface of the arm 13 on which the sensor unit 100A is attached. The reference marker portion 200A includes three reference markers 201A. The fiducial marker 201A defines a reference coordinate system. That is, the reference mark 201A is provided on the outer peripheral surface of the arm 13 so as to represent the reference coordinate system. The reference coordinate system is derived by measuring the three reference markers 201A. For example, when the three-dimensional coordinate system derived from the three reference markers 201A deviates from the reference coordinate system, it may be based on the conversion information for matching the derived three-dimensional coordinate system with the reference coordinate system, and based on the reference mark. The measurement result of the object 201A is derived from the reference coordinate system.

As a method of deriving a three-dimensional coordinate system based on three reference markers 201A, a well-known method can be used. For example, the first axis is determined by the two reference markers 201A. The face of the triangle is defined by the three reference marks 201A, and the normal of the face is used as the second axis. A line orthogonal to the two axes is used as the third axis. Further, as an example, the reference mark 201A may be a silver plate with a cross. Identification information may be attached to each of the reference markers 201A, and the reference coordinate system may be derived using the identification information.

As shown in FIG. 3(b), a reference mark portion 200B including three reference marks 201B is provided on the surface of the arm 13 on which the sensor unit 100B is attached. As shown in FIG. 3(c), a reference mark portion 200C including three reference marks 201C is provided on the surface of the arm 13 on which the sensor unit 100C is attached. As shown in FIG. 3(d), a reference mark portion 200D including three reference marks 201D is provided on the surface of the arm 13 on which the sensor unit 100D is attached. The three reference marks 201B, the three reference marks 201C, and the three reference marks 201D define a reference coordinate system in the same manner as the three reference marks 201A. In other words, the three reference marks 201A, the three reference marks 201B, the three reference marks 201C, and the three reference marks 201D define the same coordinate system (the reference coordinate system set in advance for the hydraulic excavator 1).

As shown in FIGS. 4(a) and 4(b), the sensor unit 100A includes an object detecting sensor (first object detecting sensor) 110A and a mounting jig (first mounting jig) 120A. . The object detecting sensor 110A is a sensor that detects an object around the arm 13. In the present embodiment, the object detecting sensor 110A detects a sensor (for example, LIDAR (Light Detection And Ranging)) by irradiating a plurality of beams of laser light around the arm 13.

The mounting jig 120A is a jig that mounts the object detecting sensor 110A to the arm 13. The mounting jig 120A has a sensor mounting plate 121A and four mark mounting plates 122A. An object detecting sensor 110A is mounted on the surface of the sensor mounting plate 121A. The mark mounting plate 122A is formed in a substantially L-shaped cross section. The marker mounting plate 122A is mounted to the lower surface of the sensor mounting plate 121A. A calibration marker portion 130A is provided on the marker mounting plate 122A. In the present embodiment, the calibration marker portion 130A includes eight calibration markers (first calibration markers) 131A. The calibration marker 131A is provided in each of the marker mounting plates 122A in a group of two.

The calibration marker portion 130A defines a calibration marker coordinate system (first calibration marker coordinate system). The calibration marker coordinate system is derived by measuring the calibration marker portion 130A. As a method of deriving the calibration marker coordinate system based on the calibration marker portion 130A, a well-known method can be used. For example, when the calibration marker coordinate system is derived based on the measurement results of the three calibration markers 131A, first, the first axis is determined by the two calibration markers 131A. The plane is defined by three calibration markers 131A, and the normal of the plane is used as the second axis. A line orthogonal to the two axes is used as the third axis. For example, the position of the calibration marker 131A provided at a specific position such as the left end can be used as the origin of the calibration marker coordinate system. Further, as an example, the calibration marker 131A may be a silver plate with a cross. Identification information may be attached to each of the calibration marker portions 130A, and the calibration marker coordinate system may be derived using the identification information.

As shown in FIG. 4(c), two pins 123A for positioning the object detecting sensor 110A are provided on the upper surface (sensor mounting surface) of the sensor mounting plate 121A. The object detecting sensor 110A determines the position and orientation on the sensor mounting plate 121A by the pin 123A. That is, the installation position and posture of the object detecting sensor 110A are determined by the upper surface of the sensor mounting plate 121A and the pin 123A. Thus, the upper surface of the sensor mounting plate 121A and the pin 123A serve as the sensor coordinate specific portion (the first sensor) indicating the sensor coordinate system (the first sensor coordinate system) of the object detecting sensor 110A. It functions as a coordinate-specific part.

The sensor unit 100B has the same configuration as the sensor unit 100A. Specifically, as shown in FIGS. 4( a ) and 4 ( b ), the sensor unit 100B includes an object detecting sensor (second object detecting sensor) 110B and a mounting jig (second mounting treatment). With) 120B. The mounting jig 120B has a sensor mounting plate 121B and four mark mounting plates 122B. An object detecting sensor 110B is mounted on the surface of the sensor mounting plate 121B. A calibration marker portion 130B is provided on the marker mounting plate 122B. Similarly to the calibration marker portion 130A, the calibration marker portion 130B includes eight calibration markers (second calibration markers) 131B. The calibration marker 131B is provided in each of the marker mounting plates 122B in a group of two.

The calibration marker portion 130B defines a calibration marker coordinate system (second calibration marker coordinate system). By measuring the calibration marker portion 130B, the calibration marker coordinate system is derived in the same manner as the calibration marker portion 130A. The calibration marker portion 130B has the same configuration as the calibration marker portion 130A. Further, the calibration marker coordinate system defined by the calibration marker portion 130A and the calibration marker coordinate system defined by the calibration marker portion 130B are different from each other.

As shown in FIG. 4(c), two pins 123B for positioning the object detecting sensor 110B are provided on the upper surface of the sensor mounting plate 121B. The object detecting sensor 110B determines the position and orientation on the sensor mounting plate 121B by the pin 123B. That is, the installation position and posture of the object detecting sensor 110B are determined by the upper surface of the sensor mounting plate 121B and the pin 123B. Thus, the upper surface of the sensor mounting plate 121B and the pin 123B serve as the sensor coordinate specific portion (the second sensor) indicating the sensor coordinate system (the second sensor coordinate system) of the object detecting sensor 110B. It functions as a coordinate-specific part.

The sensor units 100C and 100D have the same configuration as the sensor unit 100A. Specifically, as shown in FIG. 3( c ), the sensor unit 100C includes an object detection sensor 110C and a calibration marker unit 130C. Further, the sensor unit 100C has a mounting jig similar to the sensor unit 100A, and the mounting jig includes a sensor mounting plate provided with a pin and a mark mounting plate. As shown in FIG. 3(d), the sensor unit 100D includes an object detection sensor 110D and a calibration marker portion 130D. Further, the sensor unit 100D has a mounting jig similar to the sensor unit 100A, and the mounting jig includes a sensor mounting plate provided with a pin and a mark mounting plate.

As shown in FIG. 5, the hydraulic excavator 1 is provided with the monitoring device 20. The monitoring device 20 monitors an obstacle or the like around the arm 13 based on the detection results of the object detecting sensors 110A to 110D. Specifically, based on the detection results of the object detection sensors 110A to 110D, the monitoring device 20 grasps where an obstacle or the like exists with respect to the reference coordinate system. The monitoring device 20 performs a report of the presence or absence of an obstacle or the like based on the obstacle or the like that is grasped.

Here, in order to grasp the position of the obstacle or the like with respect to the reference coordinate system, it is necessary to previously associate the reference coordinate system with the sensor coordinate system of each of the object detecting sensors 110A to 110D. Hereinafter, a coordinate system integration method in which a reference coordinate system and a sensor coordinate system are associated will be described.

The correspondence between the reference coordinate system and the sensor coordinate system is performed by the coordinate corresponding establishing device 300 shown in FIG. The coordinate correspondence establishing device 300 may be provided in a place other than the hydraulic excavator 1 or may be mounted on the hydraulic excavator 1 .

A first measuring device 304 and a second measuring device (measuring device) 305 are connected to the coordinate correspondence establishing device 300. The first measuring device 304 measures the installation position and posture of the calibration marker portion 130A of the sensor unit 100A and the object detection sensor 110A in the sensor unit 100A. Specifically, the first measuring device 304 measures three or more calibration markers 131A as the calibration marker portion 130A. The first measuring device 304 measures the upper surface of the sensor mounting plate 121A and the pin 123A as the installation position and posture of the object detecting sensor 110A. Further, the calibration marker 131A is provided at least three calibration markers 131A at positions and orientations simultaneously measured on the upper surface of the sensor mounting plate 121A and the pin 123A.

Similarly to the other sensor units 100B to 100D, the first measuring device 304 measures the installation position and posture of the calibration marker portions 130B to 130D and the object detection sensors 110B to 110D, respectively. The first measuring device 304 inputs the measurement result to the coordinate correspondence establishing device 300. The measurement by the first measuring device 304 may be performed by an operator or automatically by the first measuring device 304. As the first measuring device 304, for example, a three-dimensional laser measuring device can be used.

Here, the measurement by the first measuring device 304 is performed in a state where the object detecting sensor 110A or the like is not mounted on the sensor mounting plate 121A or the like. In addition, the measurement of the first measuring device 304 may be performed in a state where the attachment jig 120A or the like is not attached to the arm 13, or may be performed in a state where the attachment jig 120A or the like is attached to the arm 13.

The second measuring device 305 measures the calibration marker portion 130A of the sensor unit 100A and the reference marker portion 200A provided on the arm 13 . Specifically, the second measuring device 305 measures three or more calibration markers 131A as the calibration marker portion 130A. The second measuring device 305 measures three or more reference markers 201A as the reference marker portion 200A. Further, in the present embodiment, since the reference marker portion 200A includes the three reference markers 201A, the second measuring device 305 measures all of the three reference markers 201A.

Further, when the second measuring device 305 measures the calibration marker portion 130A with respect to the reference marker portion 200A, at least three or more of the reference markers 201A are directed toward the direction to be measured together with the calibration marker portion 130A. Further, in the present embodiment, since the reference marker portion 200A includes the three reference markers 201A, all of the three reference markers 201A are oriented in the direction in which they are measured together with the calibration marker portion 130A. In other words, the three reference markers 201A and the three or more calibration markers 131A are provided so as to be viewable from one viewpoint.

Similarly to the other sensor units 100B to 100D, the second measuring device 305 measures the calibration marker portions 130B to 130D and the reference marker portions 200B to 200D provided on the arm 13 in the same manner. In the second measuring device 305, all of the three reference markers 201B are measured as the reference marker portion 200B, and all three reference markers 201C are measured as the reference marker portion 200C, and the three reference markers 201D are used as the reference. The reference marker portion 200D performs measurement.

Further, when the second measuring device 305 measures the calibration marker portion 130B with respect to the reference marker portion 200B, all of the three reference markers 201B are directed toward the direction to be measured together with the calibration marker portion 130B. When the second measuring device 305 measures the calibration marker portion 130C with respect to the reference marker portion 200C, all of the three reference markers 201C are directed toward the direction to be measured together with the calibration marker portion 130C. When the second measuring device 305 measures the calibration marker portion 130D with respect to the fiducial marker portion 200D, all of the three fiducial markers 201D are directed toward the direction to be measured together with the calibration marker portion 130D.

In this manner, the reference arm 13 is provided with three reference marks 201A, three reference marks 201B, three reference marks 201C, and three reference marks 201D, and a total of twelve reference marks. Among the reference markers, the three reference markers 201A are oriented in the direction in which they are measured together with the calibration marker portion 130A by the second measuring device 305. Similarly, among the reference markers 201A to 201D, the three reference markers 201B are oriented in the direction in which they are measured together with the calibration marker portion 130B by the second measuring device 305. Among the reference markers 201A to 201D, the three reference markers 201C are oriented in the direction in which they are measured together with the calibration marker portion 130C by the second measuring device 305. Among the reference markers 201A to 201D, the three reference markers 201D are oriented in the direction in which they are measured together with the calibration marker portion 130D by the second measuring device 305.

The second measuring device 305 inputs the measurement result to the coordinate correspondence establishing device 300. The measurement by the second measuring device 305 can be performed by an operator or automatically by the second measuring device 305. As the second measuring device 305, for example, a measuring device such as a versatile measuring instrument can be used.

The second measuring device 305 is movable. For example, the calibration marker portion 130B and the reference marker portion 200B can be measured at positions different from the positions at which the calibration marker portion 130A and the reference marker portion 200A are measured. When the second measuring device 305 is fixed to the ground, for example, the hydraulic excavator 1 may be moved or the arm 13 or the like may be swung to allow the calibration target portion and the reference marker portion to be measured to enter the measurement. Within the scope.

The coordinate correspondence establishing device 300 is physically configured to include a CPU (Central Processing Unit), a RAM (Random Access Memory) as a main memory device, and a ROM (Read Only Memory). A communication module that communicates with other devices, and a hard-working computer such as an auxiliary memory device such as a hard disk. The coordinate correspondence establishing device 300 may also include a plurality of computer units.

The coordinate correspondence establishing device 300 functionally includes a first correspondence establishing unit 301, a second correspondence establishing unit 302, and an integrating unit 303. Based on the measurement result of the first measuring device 304, the first correspondence establishing unit 301 defines the sensor coordinate system of the object detecting sensors 110A to 110D and the calibration target portions 130A to 130D, respectively, as shown in Fig. 6 . The calibration uses the marker coordinate system to establish a correspondence.

Specifically, the first correspondence establishing unit 301 calculates the sensor coordinate system of the object detecting sensor 110A based on the measurement result of the upper surface of the sensor mounting plate 121A and the pin 123A. The first correspondence establishing unit 301 calculates the calibration marker coordinate system defined by the calibration marker portion 130A based on the measurement result of the calibration marker portion 130A. The first correspondence establishing unit 301 associates the calculated sensor coordinate system of the object detecting sensor 110A with the calibration tag coordinate system defined by the calibration marker unit 130A.

Further, the first correspondence establishing unit 301 calculates the sensor coordinate system of the object detecting sensor 110B based on the measurement result of the upper surface of the sensor mounting plate 121B and the pin 123B. The first correspondence establishing unit 301 calculates the calibration marker coordinate system defined by the calibration marker portion 130B based on the measurement result of the calibration marker portion 130B. The first correspondence establishing unit 301 associates the sensor coordinate system of the calculated object detecting sensor 110B with the calibration tag coordinate system defined by the calibration marker unit 130B.

Similarly, the first correspondence establishing unit 301 calculates the calibration coordinate system of the sensor coordinate system of the object detection sensor 110C and the calibration marker unit 130C, and associates the calculated coordinate systems with each other. The first correspondence establishing unit 301 calculates the calibration coordinate system of the sensor coordinate system of the object detection sensor 110D and the calibration marker portion 130D, and associates the calculated coordinate systems with each other.

Based on the measurement result of the second measuring device 305, the second correspondence establishing unit 302 respectively displays the calibration marker coordinate system defined by the calibration marker portions 130A to 130D and the reference marker portion 200A to 200D, as shown in Fig. 6 . The defined reference coordinates are mapped.

Specifically, the second correspondence establishing unit 302 calculates the calibration marker coordinate system defined by the calibration marker portion 130A based on the measurement result of the calibration marker portion 130A. The second correspondence establishing unit 302 calculates a reference coordinate system based on the measurement result of the reference marker portion 200A. The second correspondence establishing unit 302 associates the calculated calibration marker coordinate system defined by the calibration marker portion 130A with the reference coordinate system.

Further, the second correspondence establishing unit 302 calculates the calibration marker coordinate system defined by the calibration marker portion 130B based on the measurement result of the calibration marker portion 130B. The second correspondence establishing unit 302 calculates a reference coordinate system based on the measurement result of the reference marker portion 200B. The second correspondence establishing unit 302 associates the calculated calibration marker coordinate system defined by the calibration marker portion 130B with the reference coordinate system.

Similarly, the second correspondence establishing unit 302 calculates the calibration marker coordinate system and the reference coordinate system of the calibration marker portion 130C, and associates the calculated coordinate systems with each other. The second correspondence establishing unit 302 calculates the calibration marker coordinate system and the reference coordinate system of the calibration marker portion 130D, and associates the calculated coordinate systems with each other.

Here, the first correspondence establishing unit 301 establishes a correspondence result between the corresponding correspondence establishment result and the correspondence establishment result by the second correspondence establishing unit 302, and the calibration marker coordinate system is common thereto. Therefore, the integration unit 303 establishes the corresponding sensor coordinate system and the calibration marker coordinate system by the first correspondence establishing unit 301, and the calibration marker coordinate system associated with the second correspondence establishing unit 302. The reference coordinates are integrated and the sensor coordinate system is associated with the reference coordinate system.

Specifically, the integration unit 303 establishes the result of the correspondence between the sensor coordinate system of the object detection sensor 110A and the calibration marker coordinate system defined by the calibration marker unit 130A, and the calibration mark. The corresponding calibration result of the calibration marker coordinate system and the reference coordinate system defined by the object portion 130A is integrated, and the sensor coordinate system of the object detection sensor 110A is associated with the reference coordinate system. Further, the integration unit 303 establishes the result of the correspondence between the sensor coordinate system of the object detection sensor 110B and the calibration marker coordinate system defined by the calibration marker portion 130B, and the calibration marker portion. The corresponding calibration result of the calibration marker coordinate system and the reference coordinate system defined by 130B is integrated, and the sensor coordinate system of the object detection sensor 110B is associated with the reference coordinate system.

Similarly, the integration unit 303 associates the sensor coordinate system of the object detection sensor 110C with the reference coordinate system, and associates the sensor coordinate system of the object detection sensor 110D with the reference coordinate system.

The result of the correspondence between the reference coordinate system and the sensor coordinate system performed by the coordinate correspondence establishing device 300 is input to the monitoring device 20, and is used for monitoring the obstacle or the like in the monitoring device 20.

Next, a processing flow of the coordinate system integration method performed by the coordinate correspondence establishing device 300, the first measuring device 304, and the second measuring device 305 will be described. As shown in FIG. 7, the first measuring device 304 measures the installation position and posture of the calibration marker portions 130A to 130D and the object detection sensors 110A to 110D (S101: first measurement step). The first correspondence establishing unit 301 sets the sensor coordinate system of the object detecting sensors 110A to 110D and the calibration tag coordinate system defined by the calibration marker portions 130A to 130D based on the measurement results of the first measurement step. Correspondence is established (S102: first correspondence establishment step).

The second measuring device 305 measures the calibration marker portions 130A to 130D and the reference marker portions 200A to 200D (S103: second measurement step). The second correspondence establishing unit 302 establishes the calibration marker coordinate system defined by the calibration marker portions 130A to 130D and the reference coordinate system defined by the reference marker portions 200A to 200D based on the measurement results of the second measurement step. Correspondence (S104: second correspondence establishment step). The integration unit 303 integrates the corresponding establishment result in the first correspondence establishing step with the corresponding establishment result in the second correspondence establishing step, and respectively associates the sensor coordinate system of the object detecting sensors 110A to 110D with the reference coordinate system ( S105: integration step).

As described above, according to the coordinate system integration method of the present embodiment, the first measurement device 304 is used in the first measurement step, and the positions of the calibration marker portions 130A to 130D and the object detection sensors 110A to 110D are respectively set and The posture is measured. In the second measurement step, the second measurement device 305 is used to measure the calibration marker portions 130A to 130D and the reference marker portions 200A to 200D, respectively. That is, based on the measurement results of the first measuring device 304 and the second measuring device 305, the sensor coordinate systems of the object detecting sensors 110A to 110D are associated with the reference coordinate system, and the object detecting sensors 110A to 110D are not used. The test results. Therefore, in the coordinate system integration method, the sensor coordinates of the reference coordinate system of the hydraulic excavator 1 and the object detection sensors 110A to 110D can be respectively independent of the detection accuracy of the object detection sensors 110A to 110D. Establish a correspondence.

The reference marker 201A is oriented in the direction measured by the second measuring device 305 together with the calibration marker portion 130A. Similarly, the reference marks 201B to 201D are directed to the direction measured by the second measuring device 305 together with the calibration marker portions 130B to 130D. In this case, for example, even if the object detecting sensor 110A and the object detecting sensor 110B are disposed at different positions, the calibration marker portion 130A and the reference marker 201A can be measured in the second measuring step, and The calibration marker portion 130B and the reference marker 201B can be measured. Further, in the coordinate system integration method of the present embodiment, even if a plurality of object detection sensors are provided as the object detection sensors 110A to 110D, the sensors of the respective object detection sensors 110A to 110D can be detected. The coordinate system corresponds to the reference coordinate system.

The object detecting sensors 110A to 110D detect the object by irradiating a plurality of beams of laser light around the arm 13. In consideration of this situation, for example, when the object detecting sensors 110A to 110D detect the reference marks 201A to 201D, the cases where 201A to 201D are not detected due to the difference in the irradiation interval of the laser light are not detected. However, in the coordinate system integration method of the present embodiment, since the detection result of the object detection sensors 110A to 110D can be used, the sensor coordinate system is associated with the reference coordinate system, so that the object detection detection can be performed independently. The detection accuracy of the devices 110A to 110D is established correspondingly to the coordinate system.

In the hydraulic excavator 1, for example, three reference marks 201A and three or more calibration marks 131A can be visually recognized from one place. Thereby, by measuring the calibration marker 131A and the reference marker 201A, the sensor coordinate system can be associated with the reference coordinate system. That is, the sensor coordinate system can be associated with the reference coordinate system without using the detection results of the object detection sensors 110A to 110D. Therefore, in the hydraulic excavator 1, the sensor coordinates set by the reference coordinate system of the arm 13 and the object detecting sensors 110A to 110D can be set independently of the detection accuracy of the object detecting sensors 110A to 110D. Establish a correspondence.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments. For example, although the reference mark 201A or the like is provided on the arm 13 formed in a quadrangular prism shape, the reference mark portion 200 may be provided on the cylindrical arm 15 as shown in FIG. 8 . In this case, the plurality of reference marks 201 constituting the reference marker portion 200 are respectively provided at a specific angle on the outer peripheral surface of the arm 15. A plurality of sets of fiducial markers 201 arranged at a specific angle are provided with a complex array in the axial direction of the arm 15. The fiducial marker 201 defines a reference coordinate system. As a method of deriving a three-dimensional coordinate system based on three reference markers 201, a well-known method can be used. For example, in the group of the reference markers 201 set at a specific angle, a circular surface is defined by the three reference markers 201, and the normal of the surface is used as the first axis. A straight line passing through the center of the circular surface and the specific reference mark 201 is used as the second axis. Further, a line orthogonal to the two axes may be used as the third axis. In other words, among the plurality of reference markers 201, at least three reference markers 201 arranged in the circumferential direction may be arranged so as to be measurable by the second measuring device 305.

In the embodiment, the case where the reference coordinate system is set based on the axis of the arm 13 will be described as an example. However, the reference coordinate system is not limited to being set based on the axis of the arm 13 . The reference coordinate is set at the appropriate position of the hydraulic excavator 1.

In FIG. 7, after the first measurement step (S101) and the first correspondence establishment step (S102), the second measurement step (S103) and the second correspondence establishment step (S104) are performed, but the second measurement step and the second measurement step may be performed. 2 Corresponding to the establishing step, the first measuring step and the first corresponding establishing step are performed.

Further, for example, it is known that the positional relationship between the installation position and posture of the object detection sensor 110A and the calibration marker 131A is determined by the accuracy of processing. Alternatively, the calibration marker 131A may be directly provided to the object detection sensor 110A, and the positional relationship between the installation position and posture of the object detection sensor 110A and the calibration marker 131A may be known. That is, the coordinate system defined by the calibration marker portion 130A serves as the sensor coordinate system of the object detection sensor 110A. In this case, it is not necessary to use the first measurement step of the first measuring device 304. Therefore, it is only necessary to measure the calibration marker portion 130A and the reference marker portion 200A using the second measuring device 305 (measurement step), and the integration portion 303 integrates the measurement results (integration step), so that the calibration can be performed. The sensor coordinate of the object detecting sensor 110A defined by the quasi-marker portion 130A is associated with the reference coordinate system defined by the fiducial marker portion 200A. The object detecting sensors 110B to 110D are also disposed at the position and posture of the known object detecting sensors 110B to 110D, the calibration mark 131B constituting the calibration marker portion 130B, and the calibration marker portion 130C. In the case of the positional relationship of each of the calibration markers of 130D, it is not necessary to use the first measurement step of the first measuring device 304. In this case, the sensor coordinate system of the object detecting sensors 110A to 110D may be associated with the reference coordinate system without performing the first measuring step.

Although the sensor unit 100A or the like is attached to the arm 13, the sensor unit 100A or the like may be attached to other portions than the arm 13. In addition, the hydraulic excavator 1 is used as a target device for associating the sensor coordinate system such as the object detecting sensor 110A, but is not limited to the hydraulic excavator 1. For example, a construction machine other than the unloader or the hydraulic excavator 1 or a heavy machine that performs a specific work or the like may be used as the target device.

For example, although the sensor coordinate system of the object detecting sensor 110A is specified by the upper surface of the sensor mounting plate 121A and the pin 123A, it may be based on the mark provided on the surface of the object detecting sensor 110A ( A specific sensor coordinate system such as a sensor coordinate specific portion).

The reference marker portion 200A includes three reference markers 201A, but may include four or more reference markers 201A. In this case, at least three or more of the reference markers 201A of the four or more reference markers 201A may be disposed so as to be measurable by the second measuring device 305. Further, the reference coordinate system can be derived using the three reference markers 201A measured by the second measuring device 305, and the reference coordinate system can be derived by using a known method using four or more reference markers 201A. Similarly to the reference marker portions 200B to 200D, four or more reference markers 201B to 201D may be included.

In the embodiment, an example in which the calibration marker coordinate system is derived based on the three calibration markers 131A is shown. However, the calibration marker coordinate system may be derived based on the four or more calibration markers 131A. Similarly, each of the calibration markers 131B constituting the calibration marker portion 130B and the calibration markers constituting the calibration marker portions 130C and 130D can derive the calibration marker coordinate system based on the four or more calibration markers. .

1‧‧‧Hydraulic excavator (target device, device with columnar body)

10‧‧‧Revolving body

11‧‧‧Transition body

12‧‧‧ lifting arm

13‧‧‧arm (columnar)

14‧‧‧Boiler

15‧‧‧ Arm

20‧‧‧Monitor

100A‧‧‧Sensor unit

100B‧‧‧Sensor unit

100C‧‧‧Sensor unit

100D‧‧‧Sensor unit

110A‧‧‧ Object Detection Sensor (1st Object Detection Sensor)

110B‧‧‧ Object Detection Sensor (2nd Object Detection Sensor)

110C‧‧‧ Object Detection Sensor

110D‧‧‧ Object Detection Sensor

120A‧‧‧Installation fixture (1st installation fixture)

120B‧‧‧Installation fixture (2nd installation fixture)

121A‧‧‧Sensor mounting plate (sensor specific part, first sensor coordinate specific part)

121B‧‧‧Sensor mounting plate (sensor specific part, second sensor coordinate specific part)

122A‧‧‧ mark mounting plate

122B‧‧‧Marking Mounting Plate

123A‧‧‧ Pin (sensor specific part, first sensor coordinate specific part)

123B‧‧ ‧ pin (sensor specific part, second sensor coordinate specific part)

130A‧‧‧Marking parts for calibration

130B‧‧‧ Calibration Marking Department

130C‧‧‧Marking part for calibration

130D‧‧‧ Calibration Marking Department

131A‧‧ ‧ calibration mark (first calibration mark)

131B‧‧ ‧ calibration mark (second calibration mark)

200‧‧‧ benchmark mark department

200A‧‧ Reference Marking Department

200B‧‧‧ benchmark mark department

200C‧‧‧ benchmark mark

200D‧‧‧ benchmark mark department

201‧‧‧ benchmark mark

201A‧‧‧ benchmark mark

201B‧‧‧ benchmark mark

201C‧‧‧ benchmark mark

201D‧‧‧ benchmark mark

300‧‧‧ coordinates corresponding to the establishment of the device

301‧‧‧1st Correspondence Department

302‧‧‧2nd Correspondence Establishment Department

303‧‧‧ Integration Department

304‧‧‧1st measuring device

305‧‧‧2nd measuring device (measuring device)

S101‧‧‧1st measurement step

S102‧‧‧1st correspondence establishment step

S103‧‧‧2nd measurement step

S104‧‧‧2nd correspondence establishment step

S105‧‧‧ integration steps

Fig. 1 is a view showing a schematic configuration of a hydraulic excavator according to an embodiment.

Figure 2 is a cross-sectional view taken along line II-II of Figure 1.

3(a) to 3(d) are diagrams schematically showing the sensor unit and the reference mark attached to the outer peripheral surface of the arm.

Fig. 4(a) is a perspective view showing the upper surface of the sensor unit. Figure 4(b) is a perspective view showing the lower surface of the sensor unit. Fig. 4 (c) is a perspective view showing a mounting surface of the object detecting sensor.

Fig. 5 is a block diagram showing the configuration of coordinates around the building device.

Fig. 6 is a conceptual diagram showing the correspondence relationship between coordinate systems.

Fig. 7 is a flow chart showing the processing flow of the coordinate system integration method.

Fig. 8 is a side view showing another example of the reference mark portion.

Claims (6)

A method for integrating a coordinate system, wherein a sensor coordinate system of an object detection sensor mounted on an object device and detecting an object around the target device is associated with a reference coordinate system preset to the target device, and includes: a measuring step of arranging three or more calibration markers provided on the mounting fixture that mounts the object detecting sensor to the target device and defining the sensor coordinate system by the measuring device, and The target device defines three or more reference markers of the above reference coordinate system for measurement; and And an integration step of determining, based on the measurement result of the measuring step, the sensor coordinate system defined by the calibration marker and the reference coordinate system defined by the reference marker. The coordinate system integration method of claim 1, wherein the object detection sensor comprises a first object detection sensor and a second object detection sensor; The mounting jig includes a first mounting jig that mounts the first object detecting sensor to the target device, and a second mounting jig that attaches the second object detecting sensor to the target device; The calibration marker includes three or more first calibration markers provided in the first attachment jig and defining a first sensor coordinate system of the first object detection sensor, and is provided in the second Installing the jig and defining three or more second calibration markers of the second sensor coordinate system of the second object detecting sensor; The reference marker is provided in three or more of the target devices, and when the measuring device measures three or more of the first calibration markers, at least three or more of the reference markers are oriented toward the first calibration marker. When the object is measured together, when the measuring device measures three or more of the second calibration markers, at least three or more of the reference markers are oriented in a direction to be measured together with the second calibration marker; In the measuring step, three or more of the first calibration markers and three or more reference markers are measured by the measuring device, and three or more of the second calibrations are used by the measuring device. The marker and more than three reference markers are measured; In the integration step, the first sensor coordinate system defined by the first calibration marker is associated with the reference coordinate system, and the second sensing defined by the second calibration marker is The coordinate system of the device is associated with the above reference coordinate system. A method for integrating a coordinate system, wherein a sensor coordinate system of an object detection sensor mounted on an object device and detecting an object around the target device is associated with a reference coordinate system preset to the target device, and includes: a first measuring step of sensing, by the first measuring device, a mounting jig or the object detecting sensor provided to mount the object detecting sensor to the target device and defining the sensor coordinate system a specific portion of the device and three or more calibration markers disposed on the mounting fixture and defining a calibration marker coordinate system; a first correspondence establishing step of, based on the measurement result of the first measuring step, the sensor coordinate system defined by the sensor coordinate specifying unit and the calibration mark coordinate defined by the calibration mark Establish a correspondence; a second measuring step of measuring three or more of the calibration markers by the second measuring device and three or more reference markers provided in the target device and defining the reference coordinate system; a second correspondence establishing step of correlating the calibration marker coordinate system defined by the calibration marker with the reference coordinate system defined by the reference marker based on a measurement result of the second measurement step; and And an integration step of the calibration target coordinate system and the calibration marker coordinate system established in the first correspondence establishing step, and the calibration marker coordinates established in the second correspondence establishing step And the reference coordinate system, wherein the sensor coordinate system is associated with the reference coordinate system. The coordinate system integration method of claim 3, wherein the object detection sensor comprises a first object detection sensor and a second object detection sensor; The mounting jig includes a first mounting jig that mounts the first object detecting sensor to the target device, and a second mounting jig that attaches the second object detecting sensor to the target device; The sensor coordinate specifying unit includes a first sensor coordinate set on the first mounting jig or the first object detecting sensor and defining a first sensor coordinate system of the first object detecting sensor a specific portion and a second sensor coordinate specifying portion provided in the second mounting jig or the second object detecting sensor and defining a second sensor coordinate system of the second object detecting sensor; The calibration label includes three or more first calibration markers that are provided in the first mounting fixture and define a first calibration marker coordinate system, and are provided in the second mounting fixture and define a second calibration 3 or more second calibration markers of the standard marker coordinate system; The reference marker is provided in three or more of the target devices, and when the second measuring device measures three or more of the first calibration markers, at least three or more of the reference markers are oriented toward the first school. When the second measuring device measures three or more of the second calibration markers, at least three or more of the reference markers are oriented together with the second calibration marker. Direction of measurement; In the first measuring step, the first measuring device coordinate portion and the three or more first calibration markers are measured by the first measuring device, and the first measuring device pairs the first measuring device Measuring the second sensor coordinate specific portion and three or more of the second calibration markers; In the first correspondence establishing step, the first sensor coordinate system defined by the first sensor coordinate specifying unit and the first calibration mark coordinate defined by the first calibration mark Correspondingly, the second sensor coordinate system defined by the second sensor coordinate specifying unit is associated with the second calibration mark coordinate system defined by the second calibration mark; In the second measuring step, three or more of the first calibration markers and three or more of the reference markers are measured by the second measuring device, and three of the second measuring devices are used by the second measuring device. Measuring the above second calibration marker and three or more of the above reference markers; In the second correspondence establishing step, the first calibration marker coordinate system defined by the first calibration marker is associated with the reference coordinate system defined by the reference marker, and the second coordinate The second calibration marker coordinate system defined by the calibration marker is associated with the reference coordinate system defined by the reference marker; In the above integration step, And the first calibration marker coordinate system and the first calibration marker coordinate system established in the first correspondence establishing step, and the first calibration marker coordinates established in the second correspondence establishing step. And the reference coordinate system, wherein the first sensor coordinate system is associated with the reference coordinate system, and The second calibration marker coordinate system and the second calibration marker coordinate system established in the first correspondence establishing step, and the second calibration marker coordinates corresponding to the second correspondence establishment step are established. And the reference coordinate system, wherein the second sensor coordinate system is associated with the reference coordinate system. The coordinate system integration method according to any one of claims 1 to 4, wherein the object detecting sensor detects the object by irradiating a plurality of beams of laser light to the periphery of the object device. A device having a columnar body, comprising: Movable columnar body; 3 or more reference markers, which are disposed in the columnar body, and define a reference coordinate system preset for the columnar body; An object detecting sensor that moves together with the columnar body; and 3 or more calibration markers, which are disposed on the object detection sensor or the mounting fixture mounted on the object detection sensor, and define a sensor coordinate system of the object detection sensor; Three or more of the above-mentioned reference markers and three or more of the above-described calibration markers are provided to be viewable from one location.