JP2019143995A - Construction machine position estimation device - Google Patents

Abstract

To make it possible to estimate a three-dimensional position of a construction machine without using a satellite positioning system.SOLUTION: A reference point acquisition unit 33 acquires three map reference points (Pm) in three-dimensional map information (M). A camera 35 is attached to an upper swing body 13 and acquires an image (Im) (distance image) including three existing reference points (Pr) corresponding to the three map reference points (Pm). A distance acquisition unit 41 acquires a distance (L) from each of the three existing reference points (Pr) to the camera 35 on the basis of the image (Im). A position calculation unit 43 calculates a position of a construction machine 1 with respect to the existing reference points (Pr) on the basis of the distance (L) calculated by the distance acquisition unit 41.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus for estimating a three-dimensional position of a construction machine.

  For example, Patent Document 1 describes a technique for measuring the three-dimensional position of a construction machine using a satellite positioning system (GPS (Global Positioning System in the same document)) (paragraphs 0029 and 0030 of the document). And see FIG.

JP 2009-42175 A

  However, satellite positioning systems usually used in construction machines are expensive (specific examples will be described later).

  Accordingly, an object of the present invention is to provide a position estimation device for a construction machine that can estimate the three-dimensional position of the construction machine without using a satellite positioning system.

  The construction machine position estimation apparatus of the present invention estimates the three-dimensional position of the construction machine. The construction machine includes a lower traveling body and an upper revolving body that can pivot with respect to the lower traveling body. A position estimation device for a construction machine includes a map information acquisition unit, a reference point acquisition unit, a camera, a distance acquisition unit, and a position calculation unit. The map information acquisition unit acquires three-dimensional map information. The reference point acquisition unit acquires three reference points on the map in the map information. The camera is attached to the upper swing body, and acquires a distance image including three real reference points corresponding to the three reference points on the map. The distance acquisition unit acquires a distance from each of the three real reference points to the camera based on the distance image. The position calculation unit calculates the position of the construction machine with respect to the actual reference point based on the distance calculated by the distance acquisition unit.

  With the above configuration, the three-dimensional position of the construction machine can be estimated without using a satellite positioning system.

It is a figure which shows the position estimation apparatus 20 of the construction machine 1. FIG. It is a figure which shows the map information which the position estimation apparatus 20 shown in FIG. 1 acquires. It is a figure which shows the image which the camera 35 shown in FIG. 1 image | photographed. It is a flowchart of an action | operation of the position estimation apparatus 20 shown in FIG. It is a flowchart of control of step S30 shown in FIG. It is the figure which looked at the construction machine 1 etc. which are shown in FIG. 1 from the top. It is the figure which looked at the construction machine 1 etc. which are shown in FIG. 1 from the top. FIG. 8 is a view corresponding to FIG. 7 illustrating a state in which the construction machine 1 illustrated in FIG. 7 is rotated with respect to the ground.

  With reference to FIGS. 1-8, the position estimation apparatus 20 of the construction machine 1 shown in FIG. 1 is demonstrated.

  The construction machine 1 is a machine that performs work such as construction work. The construction machine 1 is, for example, an excavator. The construction machine 1 includes a lower traveling body 11, an upper swing body 13, and an attachment 15. The lower traveling body 11 is a part that causes the construction machine 1 to travel. The lower traveling body 11 includes left and right crawlers (see FIG. 6). The upper swing body 13 is disposed above the lower travel body 11 and can swing with respect to the lower travel body 11.

  The attachment 15 is a part that is attached to the upper swing body 13 and performs work. The attachment 15 includes a boom 15a, an arm 15b, and a bucket 15c. The boom 15 a is rotatably attached to the upper swing body 13. The arm 15b is rotatably attached to the boom 15a. The bucket 15c (tip attachment) is rotatably attached to the arm 15b. Let the front-end | tip part of the attachment 15 be the attachment front-end | tip part 15t. The attachment tip 15t is, for example, a cutting edge at the tip of the bucket 15c.

  The position estimation device 20 calculates (estimates) the three-dimensional position of the construction machine 1. Hereinafter, the three-dimensional position is also simply referred to as “position”. The position estimation device 20 calculates the position of the attachment 15, for example, and calculates the position of the attachment tip 15t, for example. The position estimation device 20 includes a map information acquisition unit 31, a reference point acquisition unit 33, a camera 35, a camera rotation unit 37, a calculation unit 40, and a detection unit 50.

  The map information acquisition unit 31 acquires three-dimensional map information M. The map information acquisition unit 31 may acquire the map information M by aerial photography using a drone or the like, or may acquire known map information M stored in advance in a storage device or the like. The map information M is an electronic map including information on a site (construction site) where the construction machine 1 performs work. FIG. 2 shows an example of the map information M. Note that the map information Ma shown in the lower part of FIG. 2 is information corresponding to a state where the construction site is viewed from above. The map information Mb shown in the upper part of FIG. 2 is information corresponding to a state where a building or the like is seen in the direction of the arrow V1 shown in the map information Ma.

  The reference point acquisition unit 33 (see FIG. 1) acquires three map reference points Pm (Pm1, Pm2, and Pm3). The reference point Pm on the map is a reference point in the map information M. The position (coordinates) of the reference point Pm on the map is known. The reference point acquisition unit 33 may acquire at least three on-map reference points Pm and may acquire four or more on-map reference points Pm. Hereinafter, a case where the number of reference points Pm on the map is 3 will be described. The reference point acquisition unit 33 illustrated in FIG. 1 may acquire the reference point Pm on the map by being operated by an operator, for example, specifically, the reference point on the map by operating the touch panel or operating the cursor. The point Pm may be acquired. For example, the reference point acquisition unit 33 may acquire a map reference point Pm stored in advance in a storage device or the like. The actual reference points Pr (Pr1, Pr2, and Pr3) corresponding to the reference points Pm (Pm1, Pm2, and Pm3) on the map are set as the actual reference points Pr (Pr1, Pr2, and Pr3).

  As shown in FIG. 3, the camera 35 acquires an image Im (distance image) including three actual reference points Pr. The image Im includes information on the distance L (see FIG. 6) from each of the three real reference points Pr to the camera 35. The camera 35 may be a stereo camera, for example, a monocular camera capable of acquiring distance information, for example, a TOF (Time of Flight) camera, or the like. As shown in FIG. 1, the camera 35 is attached to the upper swing body 13.

  The camera rotation unit 37 rotates the camera 35 with respect to the upper swing body 13. The camera rotation unit 37 rotates the camera 35 in the horizontal direction (around the vertical rotation axis). The camera rotation unit 37 may rotate the camera 35 in the vertical direction (about the horizontal rotation axis).

  The computing unit 40 performs signal input / output, computation, and the like. The calculation unit 40 includes a distance acquisition unit 41, a position calculation unit 43, and an imaging range control unit 45.

  The distance acquisition unit 41 acquires a distance L (see FIG. 6) from each of the three real reference points Pr (see FIG. 6) to the camera 35 based on the image Im (see FIG. 3) acquired by the camera 35. To do. The position calculation unit 43 calculates the position (relative position) of the construction machine 1 with respect to the actual reference point Pr (see FIG. 6) based on the distance L (see FIG. 6) acquired by the distance acquisition unit 41. The photographing range control unit 45 controls the photographing range of the camera 35 so that the camera 35 continues to photograph three real reference points Pr (see FIG. 6). The shooting range control unit 45 controls the shooting range of the camera 35 in accordance with the operation of the construction machine 1. Details of the functions of the distance acquisition unit 41, the position calculation unit 43, and the imaging range control unit 45 will be described later.

  The detection unit 50 detects the state of the construction machine 1. The detection unit 50 includes a rotation detection unit 51 and an attachment posture detection unit 53. Note that the translation information acquisition unit 155 illustrated in FIG. 1 will be described in Modification 1 described later.

  The rotation detection unit 51 detects information about the rotation of the upper swing body 13 with respect to at least one of the ground and the lower traveling body 11. The “rotation information” is at least one of an angle, an angular velocity, and an angular acceleration. The rotation detection unit 51 includes a gyro sensor 51a and a turning angle detection unit 51b. The gyro sensor 51a detects the angular velocity of the upper swing body 13 with respect to the ground. The gyro sensor 51 a detects the yaw angular velocity of the upper swing body 13. The angle of the upper swing body 13 with respect to the ground may be obtained by integrating the detection value of the gyro sensor 51a with time. The turning angle detector 51 b detects the turning angle of the upper turning body 13 with respect to the lower traveling body 11. When the lower traveling body 11 is not rotating with respect to the ground, the turning angle detection unit 51b detects the angle of rotation of the upper turning body 13 with respect to the ground. The angular velocity of the upper turning body 13 with respect to the lower traveling body 11 may be obtained by differentiating the detection value of the turning angle detection unit 51b with time. For example, the turning angle detector 51 b is an angle sensor (encoder) provided in the upper turning body 13.

  The attachment posture detection unit 53 detects the posture of the attachment 15. The attachment posture detection unit 53 includes a boom angle detection unit 53a, an arm angle detection unit 53b, and a bucket angle detection unit 53c. The boom angle detection unit 53a detects the rotation angle (boom angle) of the boom 15a relative to the upper swing body 13. For example, the boom angle detection unit 53a may be an angle sensor (encoder) (the same applies to the arm angle detection unit 53b and the bucket angle detection unit 53c). For example, the boom angle detection unit 53a may detect the boom angle by detecting the amount of expansion / contraction of a cylinder (not shown) that drives the boom 15a. In this case, the boom angle is calculated from the amount of expansion / contraction of the cylinder. The same applies to the arm angle detection unit 53b and the bucket angle detection unit 53c in that the amount of expansion / contraction of the cylinder may be detected. The arm angle detector 53b detects the rotation angle (arm angle) of the arm 15b with respect to the boom 15a. The bucket angle detection unit 53c (tip attachment angle detection unit) detects the rotation angle (bucket angle) of the bucket 15c with respect to the arm 15b.

(Operation)
The operation (steps S11 to S40) of the position estimation device 20 shown in FIG. 1 will be described with reference to the flowcharts shown in FIGS. Each component of said position estimation apparatus 20 is mainly demonstrated with reference to FIG.

  In step S11 (see FIG. 4), the map information acquisition unit 31 shown in FIG. 1 acquires three-dimensional map information M (see FIG. 2).

In step S13 (see FIG. 4), the reference point acquisition unit 33 acquires three on-map reference points Pm (Pm1, Pm2, and Pm3) (see FIG. 2) in the map information M. Here, as shown in FIG. 6, the coordinates of the reference point Pm1 on the map are (X ₁ , Y ₁ , Z ₁ ), the coordinates of the reference point Pm2 on the map are (X ₂ , Y ₂ , Z ₂ ), The coordinates of the reference point Pm3 are (X ₃ , Y ₃ , Z ₃ ).

  The reference point Pm on the map is selected as follows, for example. First, the actual reference point Pr shown in FIG. 3 is selected as will be described later. Then, the position on the map information M shown in FIG. 2 corresponding to the selected real reference point Pr is selected as the map reference point Pm.

  The real reference point Pr is selected as follows. The position selected as the real reference point Pr shown in FIG. 3 is a position that can be identified from the image Im. Specifically, for example, the actual reference point Pr is selected as follows. [Example A1] The position selected as the real reference point Pr is a position that can be photographed by the camera 35 (position shown in the image Im). For example, a position hidden by a building and a position that cannot be captured by the camera 35 such as on the roof of the building are not suitable as the real reference point Pr. [Example A2] The position selected as the real reference point Pr is a position at which the real reference point Pr and other positions can be identified by the calculation unit 40 (see FIG. 1) based on the image Im. For example, a position that cannot be identified by the calculation unit 40 even in a position shown in the image Im, such as the center of a building wall, is not suitable as the real reference point Pr. [Example A3] The position selected as the real reference point Pr is a position where information (coordinate information) exists in the map information M. For example, information on the position of the corner (vertex) of the building shown in FIG. 2 and the position of the upper part of the streetlight exists in the map information M and is suitable as the actual reference point Pr. On the other hand, the window of the building shown in FIG. 3 and the column portion of the streetlight do not have information in the map information M shown in FIG. 2, and are not suitable as the real reference point Pr. [Example A4] The position selected as the real reference point Pr shown in FIG. 3 is a position at which the calculation unit 40 can easily (accurately and accurately) identify the position in the image Im (where it appears in the image Im). It is preferable that [Example A4a] For example, a position where the shape, color, etc. are easily distinguishable from surrounding objects is preferable as the actual reference point Pr. For example, in the example shown in FIG. 3, a position that can be easily distinguished from surrounding objects such as the position of the corner of the building and the position of the upper part of the streetlight is preferable as the actual reference point Pr. [Example A4b] For example, a position where an identification (image recognition) marker (mark) is attached is easy to specify the position in the image Im, and is preferable as the actual reference point Pr. [Example A4b-1] For example, the marker may be attached to a position that is easily distinguishable from surrounding objects (see [Example A4a] above). In this case, the position in the image Im can be identified more easily. [Example A4b-2] For example, even if the map information M (see FIG. 2) including the marker position information is acquired and the marker is attached to the actual position corresponding to the marker position information in the map information M Good. Further, a marker may be attached to a certain position, and information on the position where the marker is attached may be added to the map information M (see FIG. 2). In these cases, for example, even if the position is indistinguishable from surrounding objects such as the center of a building wall, the position where the marker is attached can be selected as the actual reference point Pr.

  Next, in step S15 (see FIG. 4), as shown in FIG. 3, the initial position of the shooting range of the camera 35 is set so that the three shooting reference points Pr are included in the shooting range of the camera 35. For example, the camera 35 may be rotated by the camera rotation unit 37 shown in FIG. 1 so that the shooting range of the camera 35 includes the three real reference points Pr. The orientation may be changed. For example, the imaging range of the camera 35 is set by the operator operating the direction of the camera 35 or the upper swing body 13 while looking at the image Im or the camera 35.

  Here, the position of the central part of the three actual reference points Pr is defined as a reference point center position P0. The reference point center position P0 is a position where the distances from the three actual reference points Pr are equal. As shown in FIG. 8, the direction (direction) from the camera 35 toward the reference point center position P0 is defined as a reference point center position direction D0. A direction (orientation) from the camera 35 toward the front of the camera 35 is a camera shooting direction D35. Further, the position of the central portion of the image Im shown in FIG. 3 is defined as the central portion of the image Im. The central portion of the image Im may be a central portion in the horizontal direction of the image Im and a central portion in the vertical direction of the image Im. The central portion of the image Im may be a central portion (long range in the vertical direction) in the horizontal direction of the image Im.

  In step S15 (see FIG. 4), for example, the shooting range of the camera 35 is set so that the reference point center position P0 is reflected in the center (or the vicinity thereof) of the image Im. For example, as shown in FIG. 7, the shooting range of the camera 35 is set so that the reference point center position direction D0 and the camera shooting direction D35 match (or substantially match).

  In step S21 (see FIG. 4), as shown in FIG. 3, the camera 35 captures an image Im including three real reference points Pr. The positions of the three real reference points Pr in the image Im are recognized by the calculation unit 40 (see FIG. 1) as follows. For example, the operator may recognize the position of the actual reference point Pr by the operator by designating three actual reference points Pr in the image Im. The operation is, for example, a touch panel operation or a cursor operation, and is the same operation as the operation of the reference point acquisition unit 33. Further, for example, the calculation unit 40 may automatically recognize three real reference points Pr (for example, the positions of identification markers) in the image Im.

Next, in step S23 (see FIG. 4), the distance acquisition unit 41 (see FIG. 1) acquires the distance L from each of the three real reference points Pr shown in FIG. The distance acquisition unit 41 (see FIG. 1) acquires the distance L based on the image Im (see FIG. 3). Here, the distance from the real reference point Pr1 to the camera 35 and the distance L _1. The distance from the real reference point Pr2 to the camera 35 and the distance L _2. The distance from the real reference point Pr3 to the camera 35 and the distance L _3.

Next, in step S25 (see FIG. 4), the position calculation unit 43 (see FIG. 1) calculates the position of the camera 35 with respect to the actual reference point Pr. The position calculation unit 43 calculates the position of the camera 35 based on the distance L acquired by the distance acquisition unit 41 (see FIG. 1). More specifically, the position calculation unit 43 calculates the three-dimensional coordinates (X _C , Y _{C) of the} camera 35 from the distances L ₁ , L ₂ , and L ₃ and the coordinates of the real reference points Pr 1, Pr 2, and Pr 3. , Z _C ). Specifically, the position calculation unit 43 calculates the three-dimensional coordinates of the camera 35 by solving the following simultaneous equations.

  Next, in step S27 (see FIG. 4), the position calculator 43 shown in FIG. 1 calculates the position of the attachment tip 15t. The position calculation unit 43 calculates the position of the attachment tip 15t based on the position of the camera 35, the camera rotation angle Rc (see FIG. 7), and the attitude of the attachment 15.

  More specifically, the position calculation unit 43 calculates the upper swing body direction D13 based on the reference point center position direction D0 and the camera rotation angle Rc shown in FIG. The upper swing body direction D13 is the direction (orientation) of the upper swing body 13 and is, for example, the direction in which the attachment 15 extends from the upper swing body 13. Then, based on the position of the camera 35, the upper swing body direction D13, and the attitude of the attachment 15, the position of the attachment tip 15t is calculated. As a result, it is possible to calculate a position where the work using the attachment 15 is performed (for example, a position where excavation is performed).

  Here, the reference point center position direction D0 used for the calculation in step S27 can be calculated from the position of the camera 35 calculated in step S25 (see FIG. 4) and the position of the reference point center position P0. The position of the reference point center position P0 can be calculated from the positions of three map reference points Pm (see FIG. 2).

  The camera rotation angle Rc used for the calculation in step S27 is an angle of the camera photographing direction D35 with respect to the upper swing body direction D13. The camera rotation angle Rc can be acquired by, for example, a sensor provided in the camera rotation unit 37 (see FIG. 1).

  Further, the posture of the attachment 15 used for the calculation in step S27 is detected by the attachment posture detection unit 53 shown in FIG. More specifically, the boom angle detection unit 53a detects the boom angle, the arm angle detection unit 53b detects the arm angle, and the bucket angle detection unit 53c detects the bucket angle. Also, information such as the dimensions of the attachment 15 necessary for calculating the position of the attachment tip 15t is preset in the position calculator 43 (in the calculator 40).

  Next, in step S30 (see FIG. 4), the photographing range control unit 45 controls the photographing range of the camera 35 so that the camera 35 continues to photograph the three real reference points Pr shown in FIG. The reason for this control is as follows. When the construction machine 1 shown in FIG. 1 performs work, the lower traveling body 11 translates, the lower traveling body 11 rotates with respect to the ground (rotation due to the difference in traveling speed between the left and right crawlers), and the upper traveling body 11 moves upward. At least one of the turning of the turning body 13 is performed. Then, the actual reference point Pr tends to deviate from the shooting range of the camera 35 shown in FIG. Therefore, the imaging range control unit 45 controls the imaging range of the camera 35 according to the operation of the construction machine 1 (see FIG. 1) so that the camera 35 continues to capture the three real reference points Pr. As a specific example of the control, as shown in FIG. 8, the control of the shooting range of the camera 35 when the upper swing body direction D13 is rotated by the angle θ from the direction 13a to the direction 13b will be described.

  In step S31 (see FIG. 5), the rotation detection unit 51 (see FIG. 1) detects information on the rotation of the upper swing body 13. The rotation information detected by the rotation detection unit 51 is, for example, an angle θ and an angular velocity. This detection is performed, for example, as in [Example B1] and [Example B2] below. [Example B1] When the lower traveling body 11 is not rotating with respect to the ground, this detection is performed by at least one of the gyro sensor 51a and the turning angle detector 51b shown in FIG. [Example B1a] When the angular velocity is detected by the gyro sensor 51a, the angular change amount (angle θ shown in FIG. 8) within a predetermined time may be obtained by integrating the angular velocity with time. When this calculation is performed, an error caused by integrating the detection value of the gyro sensor 51a may be corrected based on the angle detected by the turning angle detection unit 51b. [Example B1b] When an angle is detected by the turning angle detector 51b, the angular velocity may be obtained by differentiating this angle with time. [Example B2] When the lower traveling body 11 is rotating with respect to the ground, this detection is performed by the gyro sensor 51a.

  Next, in step S33 (see FIG. 5), the imaging range control unit 45 (see FIG. 1) sets the deviation angle Rs based on the amount of change (angle θ) in the upper turning body direction D13 with respect to the ground shown in FIG. Calculate (estimate). The deviation angle Rs is the magnitude of the angle deviation of the camera photographing direction D35 with respect to the reference point center position direction D0 (for example, the clockwise direction as viewed from above is positive and the counterclockwise direction is negative). For example, the deviation angle Rs is equal to the angle θ. When the upper swing body direction D13 rotates by the angle θ while the construction machine 1 is translated, the deviation angle Rs is substantially equal to the angle θ. Hereinafter, the photographing range control unit 45 will be described with reference to FIG.

  Next, in step S35 (see FIG. 5), the shooting range control unit 45 sets a target range (target shooting range) of the shooting range of the camera 35 using the deviation angle Rs. Specifically, the shooting range control unit 45 sets the target shooting range so that the shift angle Rs approaches zero. The imaging range control unit 45 sets the target angle of the camera rotation unit 37 (see FIG. 1) so that the shift angle Rs approaches zero. The imaging range control unit 45 sets the target angle of the camera rotation unit 37 so that the camera 35 rotates in the opposite direction to the shift angle Rs and by the same magnitude as the shift angle Rs.

  Next, in step S37 (see FIG. 5), the shooting range control unit 45 uses the angular velocity of the upper swing body 13 with respect to the ground (the value detected in step S31), and the target angular velocity of rotation of the shooting range of the camera 35. Set. Specifically, the imaging range control unit 45 rotates the camera so that the camera 35 rotates at an angular velocity opposite to the direction of change of the deviation angle Rs and at the same magnitude as the angular velocity of the deviation angle Rs. The target angular velocity of the moving part 37 is set.

  Next, in step S39 (see FIG. 5), the shooting range control unit 45 changes the shooting range of the camera 35 based on the target angle (set in step S35) and the target angular velocity (set in step S37). Specifically, the imaging range control unit 45 outputs a drive command to the camera rotation unit 37 (see FIG. 1) based on the target angle and the target angular velocity, and rotates the camera 35. Therefore, even when the construction machine 1 is operated, the camera 35 (camera shooting direction D35) is controlled to always face the reference point center position P0.

  Next, in S40 (see FIG. 4), it is determined whether or not the position estimation by the position estimation device 20 shown in FIG. For example, it is determined whether or not an operation for ending position estimation has been performed by the operator. When the position estimation ends, the flow ends. When the position estimation is not completed, the process returns to step S21 (see FIG. 4), the calculation of the position of the construction machine 1 (step S25, step 27 in FIG. 4) is continued, and the photographing range of the camera 35 is controlled (step S30). Will continue. Thus, in this embodiment, it is possible to keep track of the position of the construction machine 1 without using a satellite positioning system such as GPS.

(About computerized construction)
Conventionally, information-oriented construction, which is construction work based on electronic information on the three-dimensional topography of construction sites, has been performed. By computerized construction, it is planned to eliminate choking and surveying work in the middle of work. In order to carry out computerized construction, it is necessary to acquire the position of the construction machine. Therefore, conventionally, the position of the construction machine has been acquired using a satellite positioning system such as GPS. Moreover, since it is necessary to acquire a position with high accuracy in the information construction, for example, a high-accuracy satellite positioning system (for example, real-time kinematic positioning) using information of a fixed station may be used. However, a high-accuracy satellite positioning system is expensive, and is more expensive than, for example, the price of medium to small excavators.

  If only the position of the construction machine is acquired, only one receiver of the satellite positioning system is required. However, in order to perform computerized construction, it is necessary to acquire the position of the part where the work is performed, specifically, for example, the position of the tip of the attachment. For this purpose, information on the direction of the upper-part turning body with respect to the ground is necessary. Therefore, conventionally, there are cases where two receivers of the satellite positioning system are provided in the construction machine in order to acquire information on the direction of the upper-part turning body with respect to the ground. Therefore, the cost is higher than the case where only one receiver is provided. As described above, the technology for acquiring the position of the construction machine using the satellite positioning system is expensive. On the other hand, in this embodiment, the position of the construction machine 1 can be acquired (estimated) without using a satellite positioning system. Therefore, the cost can be reduced as compared with the technology using the satellite positioning system. Moreover, in this embodiment, information construction can be performed without using a satellite positioning system.

(effect)
The effects of the position estimation device 20 of the construction machine 1 shown in FIG. 1 are as follows.

(Effect of the first invention)
The position estimation device 20 estimates the three-dimensional position of the construction machine 1. The construction machine 1 includes a lower traveling body 11 and an upper revolving body 13 that can pivot with respect to the lower traveling body 11. The position estimation device 20 includes a map information acquisition unit 31, a reference point acquisition unit 33, a camera 35, a distance acquisition unit 41, and a position calculation unit 43.

  [Configuration 1] The map information acquisition unit 31 acquires three-dimensional map information M. The reference point acquisition unit 33 acquires three on-map reference points Pm in the map information M as shown in FIG. As shown in FIG. 1, the camera 35 is attached to the upper swing body 13. As shown in FIG. 3, the camera 35 acquires an image Im (distance image) including three real reference points Pr corresponding to the three map reference points Pm (see FIG. 2). The distance acquisition unit 41 (see FIG. 1) acquires the distance L from each of the three real reference points Pr to the camera 35, as shown in FIG. 6, based on the image Im (see FIG. 3). The position calculation unit 43 illustrated in FIG. 1 calculates the position of the construction machine 1 with respect to the actual reference point Pr (see FIG. 6) based on the distance L (see FIG. 6) calculated by the distance acquisition unit 41.

  In the above [Configuration 1], the camera 35 shown in FIG. 3 acquires the image Im. Then, based on the image Im, a distance L from the real reference point Pr shown in FIG. 6 to the camera 35 is acquired. Based on the distance L, the position of the construction machine 1 with respect to the actual reference point Pr is calculated. Therefore, the position (three-dimensional position) of the construction machine 1 with respect to the actual reference point Pr can be estimated without using a satellite positioning system. Usually, a device (such as the camera 35 capable of acquiring a distance image) constituting the position estimation device 20 is lower in cost than a device for using a satellite positioning system. Therefore, the cost of the position estimation device 20 can be reduced as compared with the technology using the satellite positioning system.

(Effect of the second invention)
[Configuration 2] As shown in FIG. 1, the position estimation device 20 includes a camera rotation unit 37 and an imaging range control unit 45. The camera rotation unit 37 rotates the camera 35 with respect to the upper swing body 13. The imaging range control unit 45 causes the camera rotation unit 37 to rotate the camera 35 according to the operation of the construction machine 1 so that the camera 35 continues to capture the three real reference points Pr (see FIG. 3). The imaging range of the camera 35 is controlled.

  With the above [Configuration 2], even when the construction machine 1 operates, the camera 35 can continue to photograph the three real reference points Pr (see FIG. 3). Therefore, the position estimation device 20 can continue to calculate the position of the construction machine 1 even if the construction machine 1 is operated.

(Effect of the third invention)
[Configuration 3] The position estimation device 20 includes a rotation detection unit 51. The rotation detection unit 51 detects information about the rotation of the upper swing body 13 with respect to at least one of the ground and the lower traveling body 11. The shooting range control unit 45 controls the shooting range of the camera 35 based on the information detected by the rotation detection unit 51.

  With the above [Configuration 3], even when the upper swing body 13 rotates relative to the ground or the lower traveling body 11, the camera 35 can continue to photograph the three real reference points Pr (see FIG. 3). Therefore, the position estimation device 20 can continue to calculate the position of the construction machine 1 even when the upper swing body 13 rotates.

(Modification 1)
When the construction machine 1 shown in FIG. 8 moves in translation, the camera photographing direction D35 may deviate from the reference point center position direction D0 (the displacement angle Rs increases). Therefore, the shooting range control unit 45 may control the shooting range of the camera 35 based on the translational movement information of the construction machine 1. In the present modification, the detection unit 50 illustrated in FIG. 1 includes a translation information acquisition unit 155.

  The translation information acquisition unit 155 acquires information on the translational movement of the construction machine 1 with respect to the ground. The information acquired by the translation information acquisition unit 155 is, for example, the speed (speed and direction) or acceleration (magnitude and direction of acceleration) of translation. For example, the translation information acquisition unit 155 may acquire the speed of translation or the magnitude of acceleration by detecting a command (such as an electrical command or a hydraulic command) for driving the lower traveling body 11. . For example, the translation information acquisition unit 155 determines the direction of translation of the construction machine 1 as the upper turning body direction D13 (see FIG. 8) and the angle of the upper turning body 13 with respect to the lower traveling body 11 (detection by the turning angle detection unit 51b). Value). For example, the translation information acquisition unit 155 may acquire (calculate) information about translation from the change in the position of the construction machine 1 calculated by the position calculation unit 43.

(Effect of the fourth invention)
[Configuration 4] As shown in FIG. 1, the position estimation device 20 includes a translation information acquisition unit 155. The translation information acquisition unit 155 acquires information on the translational movement of the construction machine 1 with respect to the ground. The shooting range control unit 45 controls the shooting range of the camera 35 based on the information acquired by the translation information acquisition unit 155.

  With the above [Configuration 4], even when the construction machine 1 moves in translation, the camera 35 can continue to photograph the three real reference points Pr (see FIG. 3). Therefore, the position estimation device 20 can continue to calculate the position of the construction machine 1 even if the upper swing body 13 moves in translation.

(Modification 2)
The shooting range control unit 45 may control the shooting range of the camera 35 in the vertical direction. For example, based on the change in the inclination of the construction machine 1 with respect to the horizontal plane, the camera 35 may be rotated in the vertical direction (rotated about the horizontal axis) by the camera rotation unit 37. Further, when the construction machine 1 shown in FIG. 6 approaches or moves away from the actual reference point Pr, the reference point center position direction D0 may be shifted in the vertical direction with respect to the camera photographing direction D35 shown in FIG. . Therefore, the shooting range control unit 45 may control the shooting range of the camera 35 in the vertical direction based on the translation information.

(Other variations)
The above embodiment may be variously modified. For example, the constituent elements of the above-described embodiment and each modification may be combined. For example, the order of the steps in the flowcharts shown in FIGS. 4 and 5 may be changed. For example, the number of components of the position estimation device 20 illustrated in FIG. 1 may be changed, and some of the components may not be provided.

  For example, in the above embodiment, the shooting range control unit 45 controls the shooting range of the camera 35 according to the operation of the construction machine 1. On the other hand, the calculation unit 40 recognizes the movement of the real reference point Pr in the image Im shown in FIG. 3 (image recognition), and controls the shooting range of the camera 35 so as to follow the movement of the real reference point Pr. (Following by image recognition may be performed). For example, when the change in the deviation angle Rs shown in FIG. 8 is so slow that tracking by image recognition is possible (for example, when the construction machine 1 moves in translation), tracking by image recognition may be performed. On the other hand, when the upper turning body 13 turns with respect to the lower traveling body 11, the change in the deviation angle Rs is usually so fast that tracking by image recognition is impossible. Therefore, when the upper turning body 13 turns with respect to the lower traveling body 11, it is preferable to control the imaging range of the camera 35 based on the rotation information of the upper turning body 13 detected by the rotation detection unit 51.

  For example, the camera 35 shown in FIG. 1 may be an omnidirectional camera. In this case, the camera rotation part 37 which rotates the camera 35 is unnecessary. In this case, a part including the actual reference point Pr (see FIG. 3) is cut out (trimmed) from the image Im (see FIG. 3) obtained by the camera 35, and a distance L (see FIG. 6) is obtained from the cut out part. May be. In this case, the cutout range may be controlled in accordance with the operation of the construction machine 1 in the same manner as the shooting range control by the shooting range control unit 45.

  Among the components of the position estimation device 20, the camera 35 and the camera rotation unit 37 are provided in the construction machine 1. On the other hand, among the components of the position estimation device 20, the components other than the camera 35 and the camera rotation unit 37 may be provided in the construction machine 1 or may be provided outside the construction machine 1. For example, the map information acquisition unit 31, the reference point acquisition unit 33, and the calculation unit 40 may be realized by a computer outside the construction machine 1 or the like. For example, at least a part of the detection unit 50 may be provided outside the construction machine 1, for example, a camera that detects the state of the construction machine 1 from the outside of the construction machine 1.

  At least some of the components of the position estimation device 20 may be configured integrally or may be provided separately. Specifically, for example, the map information acquisition unit 31, the reference point acquisition unit 33, and the calculation unit 40 may be realized by a single computer configured integrally. For example, at least one of the distance acquisition unit 41, the position calculation unit 43, and the shooting range control unit 45 (for example, the shooting range control unit 45) is different from the other parts (for example, the distance acquisition unit 41 and the position calculation unit 43). It may be provided separately.

DESCRIPTION OF SYMBOLS 1 Construction machine 11 Lower traveling body 13 Upper turning body 20 Position estimation apparatus 31 Map information acquisition part 33 Reference point acquisition part 35 Camera 37 Camera rotation part 41 Distance acquisition part 43 Position calculation part 45 Shooting range control part 51 Rotation detection part 155 Translation information acquisition unit Im image (distance image)
L, L ₁ , L ₂ , L ₃ distance M Map information Pm, Pm1, Pm2, Pm3 Map reference point Pr, Pr1, Pr2, Pr3 Real reference point

Claims (4)

A position estimation device for a construction machine that estimates a three-dimensional position of a construction machine comprising a lower traveling body and an upper swinging body that is capable of swiveling relative to the lower traveling body,
A map information acquisition unit for acquiring three-dimensional map information;
A reference point acquisition unit for acquiring three reference points on the map in the map information;
A camera attached to the upper swing body for obtaining a distance image including three real reference points corresponding to the three reference points on the map;
A distance acquisition unit that acquires a distance from each of the three real reference points to the camera based on the distance image;
A position calculation unit that calculates the position of the construction machine with respect to the real reference point based on the distance calculated by the distance acquisition unit;
Comprising
Construction machine position estimation device. The position estimation device for a construction machine according to claim 1,
A camera rotation unit that rotates the camera with respect to the upper swing body;
Shooting range control for controlling the shooting range of the camera by rotating the camera to the camera rotation unit according to the operation of the construction machine so that the camera continues to capture the three real reference points And
Comprising
Construction machine position estimation device. A position estimation device for a construction machine according to claim 2,
A rotation detection unit that detects rotation information of the upper swing body with respect to at least one of the ground and the lower traveling body;
The shooting range control unit controls the shooting range of the camera based on information detected by the rotation detection unit.
Construction machine position estimation device. A position estimation device for a construction machine according to claim 2 or 3,
A translation information obtaining unit for obtaining information on translational movement of the construction machine with respect to the ground;
The imaging range control unit controls the imaging range of the camera based on the information acquired by the translation information acquisition unit.
Construction machine position estimation device.

Images