EP3757299B1

EP3757299B1 - Apparatus for generating environment data around construction equipment and construction equipment including the same

Info

Publication number: EP3757299B1
Application number: EP20182034.7A
Authority: EP
Inventors: Hee Seung Hong; Dong Mok Kim; Hee Jin Lee; Hyung Do Nam; Chi Won Roh; Soo Jun Lee
Original assignee: Potenit Co Ltd; Hyundai Doosan Infracore Co Ltd
Current assignee: POTENIT CO Ltd; HD Hyundai Infracore Co Ltd
Priority date: 2019-06-25
Filing date: 2020-06-24
Publication date: 2022-03-30
Anticipated expiration: 2040-06-24
Also published as: KR20210000593A; CN112127417A; CN112127417B; EP3757299A1

Description

BACKGROUND OF THE DISCLOSURE

Field of the Disclosure

The present disclosure relates to an apparatus for generating environment data around a construction equipment and the construction equipment including the same.

Description of the Related Art

The automation of a construction equipment senses the environment around the construction equipment, and performs an automatic control by using environment data representing the surrounding environment. Such environment data may be generated by using various sensors, but since the construction equipment generally protrudes, the environment data may also include a portion of the construction equipment. Accordingly, in order to accurately represent the environment around the construction equipment as intended, data representing the portion of the construction equipment among the acquired environment data needs to be removed. Patent Literature WO 2013/170348 A1 provides an apparatus for installation on a vehicle suitable for mining excavation including at least a first and second camera configured to capture digital images of at least a portion of the mining excavation.

SUMMARY OF THE DISCLOSURE

According to the present invention, an apparatus for generating environment data around a construction equipment, and particularly, an apparatus for generating environment data representing a portion excluding a portion of the construction equipment among the environment around the construction equipment is provided, as defined by independent claim 1.
An environment data generating apparatus for generating environment data representing environment around a construction equipment according to the present invention includes a member position data generating module which generates member position data including a coordinate of a member of the construction equipment, an environment data generating module which generates first environment data representing the environment around the construction equipment, and a data processing module which generates second environment data representing the exclusion of the member of the construction equipment among the surrounding environment by converting the coordinate comprised in the first environment data into the relative coordinate to the construction equipment and excluding data representing the member of the construction equipment among the first environment data based on the member position data.
A construction equipment capable of performing construction in construction work according to exemplary embodiments of the present disclosure includes a moving part configured to move the construction equipment, a work part configured to perform construction, and an environment data generating apparatus configured to generate environment data representing environment around the construction equipment, and the environment data generating apparatus includes a member position data generating module which generates member position data including the coordinate of the member of the construction equipment, an environment data generating module which generates first environment data representing the environment around the construction equipment, and a data processing module which generates second environment data representing the exclusion of the member of the construction equipment among the surrounding environment by excluding data representing the member of the construction equipment among the first environment data based on the member position data.
A method for generating environment data representing environment around a construction equipment according to the present invention, as defined by independent claim 9, includes generating member position data including the coordinate of a member of the construction equipment, generating first environment data representing environment around the construction equipment, and generating second environment data representing the environment around the construction equipment excluded the member of the construction equipment by converting the coordinate comprised in the first environment data into the relative coordinate to the construction equipment and excluding data representing the member of the construction equipment among the first environment data based on the member.
The environment data generating apparatus and the construction equipment including the same according to the present invention generates the environment data representing only the portion excluding the construction equipment among the environment around the construction equipment, thereby reducing the subsequent processing of the environment data as well as acquiring more accurate information about the surrounding environment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an excavator, which is an example of a construction equipment according to exemplary embodiments of the present disclosure.
FIG. 2 is a diagram illustrating an environment data generating apparatus according to exemplary embodiments of the present disclosure.
FIGS. 3 to 5 are diagrams exemplarily illustrating a three-dimensional area centered on a member according to exemplary embodiments of the present disclosure.
FIG. 6 is a diagram illustrating a method for generating surrounding environment data according to exemplary embodiments of the present disclosure.
FIG. 7 is a diagram illustrating the method for generating the surrounding environment data according to exemplary embodiments of the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings.
FIG. 1 is a diagram illustrating an excavator, which is an example of a construction equipment according to exemplary embodiments of the present disclosure. Hereinafter, exemplary embodiments of the present disclosure will be described by using the accompanying drawings.
FIG. 1 is a diagram illustrating an excavator, which is an example of a construction equipment according to exemplary embodiments of the present disclosure. The construction equipment refers to equipment used in civil engineering works or building construction. Referring to FIG. 1, an excavator 1 may include a lower traveling body 10 which is movable along the ground, an upper rotating body 20 which is rotatably provided on the upper portion of the lower traveling body 10, a boom 30 which is rotatably coupled to the upper rotating body 20, a boom cylinder 40 which rotates the boom 30, an arm 50 which is rotatably coupled to the front end of the boom 30, an arm cylinder 60 which rotates the arm 50, a bucket 70 which is rotatably coupled to the front end of the arm 50, and a bucket cylinder 80 which rotates the bucket 70. Each component 10, 20, 30, 40, 50, 60, 70, and 80 of the excavator 1 may be referred to as an excavator member (hereinafter, member).
The excavator 1 may operate the boom 30 by using the boom cylinder 40, operate the arm 50 by using the arm cylinder 60, and operate the bucket 70 by using the bucket cylinder 80, thereby performing work such as digging or mowing the ground.
Meanwhile, FIG. 1 illustrates only the excavator 1 as an example of the construction equipment, but it is natural that exemplary embodiments of the present disclosure are not limited to the excavator 1, but may be applied to various construction equipments used in the construction work. However, for convenience of description, the excavator 1 will be described below as an example of the construction equipment.
FIG. 2 is a diagram illustrating an environment data generating apparatus according to exemplary embodiments of the present disclosure. Referring to FIGS. 1 and 2, an environment data generating apparatus 100 may be coupled to the excavator 1. According to exemplary embodiments, the respective components of the environment data generating apparatus 100 may be included in the excavator 1, attached to the excavator 1, or electrically connected to the excavator 1.
As described later, the environment data generating apparatus 100 may sense the environment around the excavator 1, and generate environment data representing the surrounding environment according to the sensed result. Particularly, the environment data generating apparatus 100 according to exemplary embodiments of the present disclosure may generate environment data representing only a portion excluding the excavator among the sensed surrounding environment.
The environment data generating apparatus 100 may include a sensor module 110, a member position data generating module 120, an environment data generating module 130, a data processing module 140, and a memory 150.
Hereinafter, a module mentioned in the present specification refers to hardware, software, or hardware including software which may perform at least one function. That is, the specific module described in the present specification may mean software, a device, or a circuit which may perform a corresponding function, or a device in which software capable of performing the function is executed.
The sensor module 110 may include a plurality of sensors. According to exemplary embodiments, the sensor module 110 may recognize the surrounding environment of the excavator 1, measure the position of the excavator 1, and measure a kinetic position including the position and posture of each member of the excavator 1. For example, the sensor module 110 may include at least one among a camera, a lidar, a radar, an infrared sensor, an ultrasonic sensor, a position sensor (GPS or a real time kinetic global navigation satellite system (RTK GNSS)), an intertia measurement unit (IMU) sensor, a gyro sensor, and machine guidance.
For example, the sensor module 110 may include two position sensors, and each of the two position sensors may be attached to different positions of the upper rotating body 20. Further, the sensor module 110 may include the IMU sensor, and the IMU sensor may be attached to each member 10, 20, 30, 40, 50, 60, 70, and 80 of the excavator 1. For example, the IMU sensor may be attached to the link of the excavator 1.
The sensor module 110 may generate sensing data.
The member position data generating module 120 generates member position data of the excavator 1. According to exemplary embodiments, the member position data generating module 120 may generate member position data representing the position of the member of the excavator 1 by using the information transmitted from the sensor module 110. For example, the member position data may be generated on-the-fly.
The member position data generating module 120 may calculate the relative coordinate to a reference point of each of the member of the excavator 1 by using the IMU sensor included in the sensor module 110. According to exemplary embodiments, the member position data generating module 120 may calculate a posture of the reference point of the member by using an inertia value of the member measured by the IMU sensor and the dimensions of the member of the excavator 1, and calculate the coordinate of the member by using the posture and the coordinates of the reference point of the excavator 1. At this time, the posture may be calculated as a Denavit-Hartenberg (DH) parameter.
Further, according to exemplary embodiments, the member position data generating module 120 may calculate the absolute coordinate of the reference point of the excavator 1 (for example, the center point of the excavator 1) by using the position sensor, and also calculate the absolute coordinate of the member by using the calculated absolute coordinate of the reference point and the relative coordinate of the member.
According to exemplary embodiments, the member position data generating module 120 may generate member position data including the absolute coordinate of the member. That is, the member position data includes the coordinate of the member of the excavator 1.
The environment data generating module 130 generates first environment data representing the environment around the excavator 1. According to exemplary embodiments, the first environment data may include information about an object or a terrain which is positioned around the excavator 1. For example, the first environment data may include information about coordinates, colors, sizes, and the like of the objects around the excavator 1.
Further, the first environment data may be generated in real time.
According to exemplary embodiments, the environment data generating module 130 may measure the environment around the excavator 1 by using at least one among a camera, a lidar, a radar, an infrared sensor, and an ultrasonic sensor, and calculate the coordinate of the object or the terrain which is positioned around the excavator 1 according to the measured result, and generate the first environment data including the coordinate of the object or the terrain. The coordinate may also be the absolute coordinate, but may also be the relative coordinate to the reference point of the excavator 1.
Meanwhile, since the construction equipment such as the excavator 1 has a shape which is not regular and also has a protruded portion, the first environment data generated by the environment data generating module 130 may also include information about the member of the excavator 1. That is, not only the environment around the excavator 1 measured by the sensor module 110 but also the member (for example, boom, arm, or bucket) of the excavator 1 may also be measured. In this case, since the information about the excavator 1 is not information about the intended surrounding environment, the information needs to be removed.
For example, the member of the excavator 1 may be a front work part, and the front work part not only forms a dynamic trajectory which continuously moves during the excavation, but also protrudes outward from the excavator, thereby being recognized as environment data by the sensor module 110. In this case, in a trajectory planner step of generating a trajectory of the excavator 1, this front work part may also act as noise to interfere with a trajectory planner algorithm. Accordingly, by removing data corresponding to the member such as the front work part of the excavator 1 upon generating the environment data and establishing the excavator trajectory plan in advance, it is possible to shorten the time required to generate the environment data and establish the plan.
Further, in the case of converting a Point Cloud Data (PCD) type of cognitive data into other types of data, by removing unnecessary data corresponding to the member such as the front work part, it is possible to optimize the storage, calculation time, and capacity of large-scale cognitive data. For example, even when the PCD needs to be converted into Mesh Data or a Global coordinate system, the calculation time may be reduced according to the removal process, and the storage space may be optimally used.
The data processing module 140 may generate second environment data representing only the portion excluding the excavator 1 or a portion of the excavator 1 among the environment around the excavator 1. That is, the second environment data does not include information about all or parts of the excavator 1.
The second environment data may be generated in real time.
The data processing module 140 generates the second environment data from the first environment data by using the first environment data and the member position data. According to exemplary embodiments, the data processing module 140 generates the second environment data by excluding the portion representing the member of the excavator 1.
According to exemplary embodiments, the data processing module 140 may generate the second environment data by excluding, from the first environment data, data having the coordinate adjacent to the coordinate included in the member position data among the first environment data. For example, the data processing module 140 may generate the second environment data by selecting a position value included in the member position data and a position value within a reference range among position values included in the first environment data, and removing the data corresponding to the selected position value from the first environment data.
According to exemplary embodiments, the data processing module 140 may define a three-dimensional area centered on the member based on the member position data. The data processing module 140 may generate the second environment data by removing, from the first environment data, data having a position value included in the area among the first environment data. For example, the three-dimensional area may be a cuboid shape or a sphere shape, but is not limited thereto.
For example, if the member of the excavator 1 is the front work part, the data processing module 140 may acquire the posture information of the front work part from the angles or coordinate values of the boom, the arm, and the bucket, and define the three-dimensional area of the boom, the arm, and the bucket from the acquired posture information. Further, in order to shorten the calculation time of the three-dimensional area of the front work part having a complicated shape, the data processing module 140 may calculate the coordinate values of the three-dimensional area of the front work part by simplifying the three-dimensional area to a rigid body with the three-dimensional effect. Further, the data processing module 140 may calculate the coordinate values of the three-dimensional area by using a kinematic equation from the angles of the boom, the arm, and the bucket.
Further, the data processing module 140 may generate the second environment data by selecting portion data representing an object positioned closer than the outermost member of the excavator 1 from the excavator 1 among the first environment data, and excluding, from the portion data, the data having the position value adjacent to the position value included in the member position data in order to save resources. For example, the outermost member of the excavator 1 may be a tip of the bucket 70 or a joint portion between the boom and the arm, but is not limited thereto.
Further, the data processing module 140 may generate the second environment data from the first environment data according to an absolute coordinate system or a relative coordinate system (with respect to the excavator 1). For example, based on the relative coordinate system, the data processing module 140 generates the second environment data from the first environment data by converting the coordinate of the surrounding objects included in the first environment data into the relative coordinate to the reference point of the excavator 1, and through the aforementioned process after the conversion.
Accordingly, the environment data generating apparatus 100 according to exemplary embodiments of the present disclosure may generate the environment data representing only the portion excluding the construction equipment among the environment around the construction equipment, thereby reducing the subsequent processing for the environment data as well as acquiring more accurate information about surrounding environment.
The memory 150 may store data necessary for operating the environment data generating apparatus 100. According to exemplary embodiments, the memory 150 may include information about the dimension of the excavator 1. At this time, the member position data generating module 120 may calculate the coordinate of the member of the excavator 1 based on the output from the IMU sensor and the dimensions of the excavator 1.
FIGS. 3 to 5 are diagrams exemplarily illustrating a three-dimensional area centered on the member according to exemplary embodiments of the present disclosure.
Referring to FIG. 3, the data processing module 140 may define a three-dimensional area (REG) which is formed based on the boom 30, and calculate the coordinate values of the three-dimensional area (REG). According to exemplary embodiments, the data processing module 140 may simplify the three-dimensional area (REG) centered on the boom to a rigid body with the three-dimensional effect, and calculate the coordinate values of the three-dimensional area (REG) from the simplified rigid body. For example, the data processing module 140 may define the three-dimensional area (REG) including points which are spaced by at least a predetermined distance apart from each coordinate of the boom 30 while including the boom 30. Although FIG. 3 exemplarily illustrates the three-dimensional area (REG) having a cuboid shape, exemplary embodiments of the present disclosure are not limited thereto.
Referring to FIG. 4, the data processing module 140 may define the three-dimensional area (REG) which is formed with respect to the arm 50, and calculate the coordinate values of the three-dimensional area (REG). According to exemplary embodiments, the data processing module 140 may simplify the three-dimensional area (REG) centered on the arm 50 to the rigid body with the three-dimensional effect, and calculate the coordinate values of the three-dimensional area (REG) from the simplified rigid body. For example, the data processing module 140 may define the three-dimensional area (REG) including points which are spaced by at least a predetermined distance apart from each coordinate of the arm 50 while including the arms 50. Although FIG. 4 exemplarily illustrates the three-dimensional area (REG) having a cuboid shape, exemplary embodiments of the present disclosure are not limited thereto.
Referring to FIG. 5, the data processing module 140 may define the three-dimensional area (REG) which is formed with respect to the bucket 70, and calculate the coordinate values of the three-dimensional area (REG). According to exemplary embodiments, the data processing module 140 may simplify the three-dimensional area (REG) centered on the bucket 70 to the rigid body with the three-dimensional effect, and calculate the coordinate values of the three-dimensional area (REG) from the simplified rigid body. For example, the data processing module 140 may define the three-dimensional area (REG) including points which are spaced by at least a predetermined distance apart from the coordinate of the bucket 70 while including the bucket 70. In this case, the three-dimensional area (REG) may exist outside the bucket 70, and exist on each surface of the bucket 70. For example, the three-dimensional area (REG) may include at least one surface configuring the bucket 70. Although FIG. 3 exemplarily illustrates the three-dimensional area (REG) having a cuboid shape, exemplary embodiments of the present disclosure are not limited thereto.
FIG. 6 is a diagram illustrating a method for generating surrounding environment data according to exemplary embodiments of the present disclosure. Referring to FIG. 6, the environment data generating apparatus 100 generates member position data of members of a construction equipment (S110).
The environment data generating apparatus 100 generates first environment data representing the environment around the construction equipment (S120).
The environment data generating apparatus 100 generates second environment data representing surrounding environment which excludes the member based on the member position data and the first environment data (S130).
FIG. 7 is a diagram illustrating the method for generating the surrounding environment data according to exemplary embodiments of the present disclosure. Referring to FIG. 7, the environment data generating apparatus 100 generates member position data of members of a construction equipment (S210).
The environment data generating apparatus 100 generates first environment data representing the environment around the construction equipment (S220).
The environment data generating apparatus 100 may define a three-dimensional area centered on the member based on the member position data (S230). According to exemplary embodiments, when there exist portion data representing an object positioned closer than the outermost member of the construction equipment from the construction equipment among the first environment data, the environment data generating apparatus 100 may define the three-dimensional area.
The environment data generating apparatus 100 may generate second environment data by removing, from the first environment data, data having a position value included in the three-dimensional area among the first environment data (S240).
The environment data generating apparatus and the method for generating the environment data according to exemplary embodiments of the present disclosure may be implemented by instructions stored in a computer-readable storage medium and executed by a processor.
The storage medium may store a relational database, a non-relational database, an in-memory database, or data regardless of whether it is direct and/or indirect, or whether it is in a raw state, a formatted state, an organized state, or any other accessible state, and include a database which includes a distributed type, such as other suitable databases capable of allowing access to such data through a storage controller. Further, the storage medium may include a primary storage, a secondary storage, a tertiary storage, an offline storage, a volatile storage, a nonvolatile storage, a semiconductor storage, a magnetic storage, an optical storage, a flash storage, a hard disk drive storage, a floppy disk drive, a magnetic tape, or any type of storage such as other suitable data storage medium.
In the present specification, the instructions may be any one among assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or a source code or an object code written in any combination of one or more programming languages including an object-oriented programming language such as Smalltalk or C ++, and conventional procedural programming languages such as a "C" programming language or similar programming languages.

Claims

An environment data generating apparatus (100) for generating environment data representing environment around a construction equipment, the environment data generating apparatus (100) comprising:
a member position data generating module (120) which generates member position data comprising a coordinate of a member of the construction equipment;

an environment data generating module (130) which generates first environment data representing the environment around the construction equipment; and

a data processing module (140) which generates second environment data representing the environment around the construction equipment excluded the member of the construction equipment by converting the coordinate comprised in the first environment data into the relative coordinate to the construction equipment and excluding data representing the member of the construction equipment among the first environment data based on the member position data.
The environment data generating apparatus (100) of claim 1,
wherein the member position data generating module (120) calculates a posture of the construction equipment to a reference point by using an IMU sensor or an angle sensor attached to the construction equipment and generates the member position data based on the posture.
The environment data generating apparatus (100) of claim 1,
wherein the member comprises at least one of a boom (30), an arm (50), and a bucket (70).
The environment data generating apparatus (100) of claim 1,
wherein the environment data generating module (130) generates the first environment data by using at least one of a lidar sensor, a GPS sensor, and a camera.
The environment data generating apparatus (100) of claim 1,
wherein the data processing module (140) generates the second environment data by excluding, from the first environment data, data having the coordinate adjacent to the coordinate comprised in the member position data among the first environment data.
The environment data generating apparatus (100) of claim 1,
wherein the data processing module (140) selects portion data, among the first environment data, representing an object positioned closer than the outermost member of the construction equipment from the construction equipment, and generates the second environment data by excluding, from the first environment data, data representing the member of the construction equipment among the portion data.
The environment data generating apparatus (100) of claim 1,
wherein the data processing module (140) generates the second environment data by defining a three-dimensional area centered on the member based on the member position data, and excluding, from the first environment data, data having a position value comprised in the three-dimensional area among the first environment data.
The environment data generating apparatus (100) of claim 7,
wherein the three-dimensional area has a cuboid shape.
A method for generating environment data representing environment around a construction equipment, the method comprising:
generating (S110) member position data comprising a coordinate of a member of the construction equipment;

generating (S120) first environment data representing environment around the construction equipment; and

generating (S130) second environment data representing the environment around the construction equipment excluded the member of the construction equipment by converting the coordinate comprised in the first environment data into the relative coordinate to the construction equipment and excluding data representing the member of the construction equipment among the first environment data based on the member position.