KR102095842B1 - Apparatus for Building Grid Map and Method there of - Google Patents

Abstract

The present invention discloses an apparatus and method for generating a grid map, in particular, categorizing a point cloud into different update cycles, and generating a grid map using the categorized point cloud. The grid map generating apparatus of the present invention includes a point cloud generator for generating different point clouds according to the attitude information of the sensor by using a sensor for measuring spatial information representing a 3D space including at least one object; And the generated point cloud is categorized according to different update cycles using the sensor's posture information and the spatial information, and the occupancy of the grid cells generated by considering coordinates of the categorized point cloud in the three-dimensional space. A map generator for updating the grid with the different update cycles to generate a grid map; It includes.

Description

Apparatus for Building Grid Map and Method there of}

Technical field to which an embodiment of the present invention pertains relates to an apparatus for generating a grid map. More specifically, it relates to an apparatus and method for generating a grid map used in computer graphics and vision technology.

Unlike cameras that recently provided image information, spatial information in a specific space containing an object is measured using a camera including a sensor such as infrared rays, and based on the measured spatial information, a space for an object or an obstacle is included. The technique of mapping is being actively researched. In particular, with the development of various 3D distance sensors, the reconstruction technology of the 3D environment has undergone many developments, and many Kinect style sensors from Microsoft, which provide high-definition depth information, are used.

In order to reconstruct the environment of the 3D space and generate a map, 3D position recognition and map building (SLAM) technology of the sensor is required, and 6 degrees of freedom of the sensor is used to recognize the position of the sensor. There are largely feature point-based and direct methods in the technology.

In order to build a 3D map through the 6-degree of freedom posture of the sensor, a voxel map that expresses a 3D environment as a collection of small cubes, such as a 2D grid map, is frequently used. Methods that make it compressible are widely used. The constructed 3D map is widely used in the field of computer vision, and particularly, it can be widely used for autonomous driving of a robot in a home or industry.

However, in order to reconstruct the 3D map, unlike the 2D map generation technology, there is a disadvantage in that a large amount of data is required. Therefore, there is a need to develop a technique for effectively updating the 3D spatial map according to the position or time of the sensor.

Korean Registered Patent No. 10-1415297 (Registration Date: 2014.06.27.)

The present invention has been devised to solve the above problems, and discloses an apparatus for generating a grid map for reconstructing an environment using a 3D sensor.

In particular, an apparatus for effectively generating a three-dimensional grid map using a distance-dependent grid map update method is disclosed.

The present invention has been devised to achieve the above object, and the grid map generating apparatus of the present invention uses a sensor for measuring spatial information representing a three-dimensional space including at least one object according to the attitude information of the sensor. Point cloud generation unit for generating different point clouds; And the generated point cloud is categorized according to different update cycles using the sensor's posture information and the spatial information, and the occupancy of the grid cells generated by considering coordinates of the categorized point cloud in the three-dimensional space. A grid map generator that updates grids with the different update cycles to generate grid maps; It includes.

In the present invention, the grid map generating apparatus comprises: a posture information calculation unit that calculates posture information regarding the position and direction of the sensor by performing image-based SLAM based on the measured spatial information; Further comprising, the grid map generating unit may generate the grid map using the calculated attitude information.

In the present invention, the posture information includes first coordinate information about the position of the sensor and second coordinate information about the direction of the sensor, and the spatial information is measured based on the color information of the three-dimensional space and the sensor. It may include the depth information of the three-dimensional space.

In the present invention, the posture information calculating unit is a feature point extracting unit for extracting feature points in consideration of a change in pixel values for each pixel in an image image generated using the spatial information; Further comprising, it is possible to calculate the attitude information of the sensor using the extracted feature points.

In the present invention, the posture information calculating unit is a feature vector generator for generating a feature vector based on the coordinates of the extracted feature points as an end point and the location of the sensor as an origin; Further comprising, it is possible to calculate the attitude information of the sensor using the generated feature vector.

In the present invention, the posture information calculating unit re-projects the feature vectors generated differently according to the time of measuring the posture information of the sensor, the spatial information representing the three-dimensional space, and the spatial information, and between the vectors. A correction unit determining the similarity and correcting the attitude information of the sensor using the determined similarity between vectors; Further comprising, the calculated posture information may include the corrected posture information.

In the present invention, although there is no particular limitation on a method for determining similarity between vectors by re-projecting the feature vectors, the correction unit may determine it by calculating the cosine distance between feature vectors, and besides the cosine distance, the Euclidean distance (Euclidean distance). ) Mahalanobis distance can also be used for similarity determination between vectors.

In the present invention, the grid map generation unit sets an update cycle for the number of points in at least some areas included in the point cloud required to update the occupancy of the grid cells based on the attitude information of the sensor and the spatial information. Update cycle setting unit; In addition, the grid map may be generated using a point cloud categorized according to the set update cycle.

In the present invention, the grid map generation unit includes a categorization unit that categorizes the generated point cloud into at least one group using the different update cycles; Further comprising, the grid map may be generated using the categorized point cloud.

In the present invention, the grid map generation unit searches for grid cells that search for grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud using sweep lines directed according to the first and second coordinate information. part; In addition, the grid map may be generated by updating the occupancy of the searched grid cells.

In the present invention, the categorization unit includes a filtering unit that filters points in at least a partial region of the categorized point cloud using different update periods for each group of the categorized point cloud; Further comprising, it is possible to categorize the point cloud based on the points of the filtered at least some areas.

In the present invention, the grid map generation unit calculates the occupancy of the grid cells by using the occupancy strength of points in the categorized point cloud swept on the sweep line, and the grid cells according to the increase or decrease of the logarithm of the calculated occupancy. Occupancy update unit for updating the occupancy of the; In addition, the grid map may be generated by using the grid cells in which the occupancy is updated.

In the present invention, the update period setting unit quantizes a distance to at least one point in the point cloud measured based on the sensor using the first coordinate information, the second coordinate information, and the depth information, thereby classifying a distance. A classification value generator for generating a; Further comprising, it is possible to set the update cycle using the generated classification value.

In the present invention, the grid map generator comprises an integrated unit that sets and integrates different weights in consideration of the first coordinate information, the second coordinate information, and the depth information in the grid cell in which the occupancy is updated; The grid map may be generated using the grid cells integrated with the different weights.

In addition, in order to achieve the above object, the grid map generation method performed by the grid map generation apparatus of the present invention uses the sensor to measure the spatial information representing the 3D space including at least one object to the attitude information of the sensor. Generating different point clouds accordingly; Calculating posture information regarding the position and direction of the sensor by performing image-based SLAM based on the measured spatial information; And the generated point cloud is categorized according to different update cycles using the sensor's posture information and the spatial information, and the occupancy of the grid cells generated by considering coordinates of the categorized point cloud in the three-dimensional space. Updating to the different update cycles to generate a grid map; It includes.

In the present invention, the posture information includes first coordinate information about the position of the sensor and second coordinate information about the direction of the sensor, and the spatial information is measured based on the color information of the three-dimensional space and the sensor. It may include the depth information of the three-dimensional space.

In the present invention, the generating of the grid map may include an update cycle for the number of points in at least some areas included in the point cloud required to update the occupancy of the grid cells based on the posture information of the sensor and the spatial information. Setting it; In addition, the grid map may be generated using a point cloud categorized according to the set update cycle.

In the present invention, generating the grid map may include categorizing the generated point clouds into at least one group using the different update cycles; Further comprising, the grid map may be generated using the categorized point cloud.

In the present invention, the generating of the grid map comprises searching for grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud using sweep lines directed according to the first coordinate information and the second coordinate information. step; In addition, the grid map may be generated by updating the occupancy of the searched grid cells.

In the present invention, the categorizing may include filtering points of at least some areas of the categorized point cloud using different update periods for each group of the categorized point cloud; Further comprising, it is possible to categorize the point cloud based on the points of the filtered at least some areas.

In the present invention, generating the grid map comprises calculating the occupancy of the grid cells using the occupancy strength of points in the categorized point cloud swept on the sweep line, and according to the increase or decrease of the logarithm of the calculated occupancy. Updating the occupancy of the grid cells; In addition, the grid map may be generated by using the grid cells in which the occupancy is updated.

In the present invention, the step of setting the update cycle is by quantizing a distance to at least one point in the point cloud measured based on the sensor using the first coordinate information, the second coordinate information, and the depth information. Generating a distance classification value; Further comprising, it is possible to set the update cycle using the generated classification value.

The present invention also discloses a computer program stored in a computer-readable recording medium for executing the grid map generation method described above on a computer.

The present invention can effectively generate a 3D map.

In particular, by using a point cloud with different update cycles, a grid map can be quickly generated.

1 is a block diagram of a grid map generating apparatus according to an embodiment of the present invention.
2 is an enlarged block diagram of a map generating unit in the embodiment of FIG. 1.
3 is an enlarged block diagram of an update period setting unit in the embodiment of FIG. 2.
4 is an enlarged block diagram of a categorization unit in the embodiment of FIG. 2.
5 is a flowchart of a grid map generating method according to an embodiment of the present invention.
FIG. 6 is an enlarged flow chart of the step of generating a grid map in the embodiment of FIG. 5.
7 is a flowchart of a method for generating a grid map according to another embodiment of the present invention.
8 is an enlarged flow diagram of steps of updating and generating a DDU-based voxel map in the embodiment of FIG. 7.
9 is an exemplary view showing an RGB image measured by a sensor for measuring a three-dimensional space of the present invention.
10 is an exemplary view showing a depth image of a 3D space measured by a sensor for measuring a 3D space of the present invention.
11 is an exemplary diagram showing a grid map generated by a grid map generating apparatus based on posture information and spatial information of a sensor measured at a specific time at an arbitrary sensor position.
12 is an exemplary diagram illustrating an empty grid map generated by a grid map generating apparatus according to an embodiment.
13 is an exemplary view showing a grid map generated by a grid map generating apparatus according to another embodiment of the present invention.
14 is an exemplary view showing a grid map generated by a grid map generating apparatus according to another embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

In the description with reference to the accompanying drawings, the same or corresponding components are assigned the same reference numbers and redundant description thereof will be omitted.

In addition, in describing the present invention, when it is determined that detailed descriptions of related well-known configurations or functions may obscure the subject matter of the present invention, detailed descriptions thereof may be omitted.

Terms used in the present application are only used to describe specific embodiments, and are not intended to limit the terms. Singular expressions include plural expressions unless the context clearly indicates otherwise. Each of the steps described below may be provided with one or several software modules or may be implemented with hardware in charge of each function, or a combination of software and hardware. The specific meaning and examples of each term will be described below in the order of each drawing. Hereinafter, the configuration of the grid map generating apparatus 10 according to an embodiment of the present invention will be described in detail with reference to related drawings.

1 is a block diagram of a grid map generating apparatus 10 according to an embodiment of the present invention.

The grid map generating apparatus 10 of the present invention includes a point cloud generating unit 100, an attitude information calculating unit 200, and a map generating unit 300. For example, the grid map generating device 10 generates a point cloud using a sensor that measures spatial information representing a 3D space, calculates the attitude information of the sensor, and uses the attitude information and spatial information of the sensor. The grid map may be generated by updating the occupancy of the grid cells generated in consideration of coordinates in the three-dimensional space of the point cloud at different periods.

For example, in the present invention, a sensor for measuring spatial information representing a 3D space used by the grid map generating apparatus 10 is a Kinect, RGB-D camera of Microsoft Corporation, which provides high-definition depth information. Includes all sensors capable of measuring depth and color information in 3D space, such as Lidar and TOF (Time Of Flight). Lidar detects an object by emitting ultraviolet visible light or near-infrared light as a pulsed laser, measures the reflected laser pulse, detects the distance, and Kinect uses an infrared sensor and an RGB image sensor and uses structured light. To detect objects. The grid map generating apparatus 10 may acquire a depth and RGB image using a sensor capable of measuring 3D spatial information, generate attitude information and a point cloud of the sensor from the generated depth and RGB image, A grid map may be generated by updating the occupancy of the grid map using the generated point cloud and the attitude information of the sensor.

The grid map generated by the grid map generating apparatus 10 is a virtual map implemented in a 3D space, and includes a voxel map describing whether an object exists in the 3D space in a virtual grid space. The voxel is a volume element, and is represented by a three-dimensional unit grid for representing a three-dimensional space, and preferably may be provided as a unit grid having a width, height, and height of 10 cm. In addition, the size of the grid map generated by the grid map generating apparatus 10 of the present invention may be provided in 4m, 8m, and 3m in width, height, and height, respectively. As described above, the grid map generating apparatus 10 can effectively generate a grid map in a shorter time than a conventional grid map generating technique.

The point cloud generation unit 100 generates different point clouds according to the attitude information of the sensor using a sensor that measures spatial information representing a 3D space including at least one object. The spatial information used by the point cloud generating unit 100 includes color information of a 3D space and depth information of a 3D space measured based on the sensor, and the attitude information of the sensor is first coordinate information about the position of the sensor And second coordinate information regarding the direction of the sensor.

For example, the point cloud generated by the point cloud generator 100 expresses a three-dimensional space as a collection of countless points (points), each point being the color of an object measured based on a sensor, The distance information and at least one of angle information directed to a corresponding point in the sensor include information, and a point cloud formed by a set of points represents a spatial configuration. The point cloud generated by the point cloud generator 100 may be expressed as Equation 1 below.

는 센서에서 해당 포인트를 바라본 각도 및 T는 포인트 클라우드 내 거리 및 각도 성분을 포함하는 행렬의 전치를 의미한다. 포인트 클라우드 생성부(100)는 3차원 공간을 나타내는 공간 정보들을 이용하고, 상기 센서의 자세 정보에 따라 상기 수학식 1로 표현되는 포인트들의 3차원 집합인 포인트 클라우드를 생성할 수 있다. Where Z _t represents a point in the point cloud measured at time t, r is the distance from the sensor to the point,

Is the angle of the sensor looking at the point and T is the transpose of the matrix containing distance and angle components in the point cloud. The point cloud generation unit 100 may use spatial information representing a 3D space, and generate a point cloud that is a 3D set of points represented by Equation 1 according to the attitude information of the sensor.

The posture information calculation unit 200 includes a feature point extraction unit 220, a feature vector generation unit 240, and a correction unit 260.

The feature point extracting unit 220 extracts feature points in consideration of a change amount of pixel values for each pixel in an image image generated using the spatial information. For example, the feature point extracting unit 220 selects an arbitrary pixel from an image image generated using spatial information, measures pixel values of pixels adjacent to the selected arbitrary pixel, and compares it with the arbitrary pixel. Feature points can be extracted by calculating the amount of change in pixel values.

In addition, the image image used by the feature point extracting unit 220 of the present invention may vary depending on the time of measuring the posture information of the sensor, the spatial information representing the 3D space, and the spatial information, and thus, the feature point extracting unit 220 May extract different feature points according to the time of measuring the attitude information of the sensor, the spatial information representing the 3D space, and the spatial information. For example, the feature point extracting unit can extract a feature point using a FAST keypoint, SIFT, and SURF algorithm based on a corner point, and the Visual Descriptor Binary Robust Independent Elementary Feature (BRIEF) in extracting the feature point ) Based ORB (Oriented Fast and Rotated BRIEF) Detector can be used.

The posture information of the sensor calculated by the posture information calculator 200 may include posture information of the corrected sensor, and may calculate posture information of the sensor for generating a grid map using the corrected posture information. The posture information calculating unit 200 may perform image-based SLAM based on the measured spatial information to calculate posture information regarding the position and direction of the sensor. For example, the attitude information of the sensor calculated by the attitude information calculating unit 200 may include first coordinate information about the position of the sensor and second coordinate information about the direction of the sensor. The first coordinate information calculated by the posture information calculating unit 300 may be expressed as [x, y, z] as coordinates indicating the position of the sensor in the three-dimensional orthogonal coordinate system, and the second coordinate information is the three-dimensional direction of the sensor. It can be expressed as [q0, q1, q2, q3] = [ q ] as the quaternion. Here, q0 is a real number, and q1, q2, and q3 mean complex components.

The posture information calculating unit 200 of the present invention calculates at least one key frame as a node in a path in which a sweep line emitted from a sensor is swept, and a feature point extracted from the calculated key frame and a current key frame The posture information of the sensor can be corrected by minimizing the re-projection error with the feature point. For example, the image-based SLAM used by the posture information calculating unit 200 has a keyframe as a node, and a graph-optimized SLAM (Graph-SLAM) method or Rao that makes a re-projection error of a visual feature point small. -Filtering SLAM using Blackwellized Particle Filter (RBPF) can be used.

The map generation unit 300 includes an update period setting unit 320, a categorization unit 340, a grid cell search unit 360, an occupancy degree updating unit 380, and an integration unit 390. For example, the map generation unit 300 categorizes the generated point cloud according to different update cycles using the sensor's posture information and the spatial information, and the categorized point cloud in the three-dimensional space. The grid map is generated by updating the occupancy of the grid cells generated in consideration of the coordinates with the different update cycles. For example, the map generation unit 300 generates a grid map based on the first cloud information generated by the point cloud and the attitude information calculation unit 200 and the second coordinate information generated by the point cloud generation unit 100. do.

For example, the map generation unit 300 searches for a grid cell corresponding to coordinates in which the point cloud is located in 3D space using a ray-casting technique that projects a ray to the point cloud, and the searched grid Update the cell occupancy to generate a grid map. The grid map generated by the map generator 300 is a set of voxels having a volume in 3D space, and includes a voxel map as described above.

The update period setting unit 320 includes a classification value generation unit 322. For example, the update period setting unit 320 updates on the number of points in at least some areas included in the point cloud required to update the occupancy of the grid cell based on the posture information of the sensor and the spatial information. Set the period as a distance classification function. By using the update period set in the update period setting unit 320, the map generation unit 300 generates a grid map quickly because the number of points in the point cloud required to update the occupancy of grid cells in the grid map can be adjusted. can do.

For example, the update period setting unit 320 may be determined based on the sensor based on the first coordinate information, the second coordinate information, and depth information up to an arbitrary point in the point cloud measured based on the sensor. The distance to the point can be quantized, and an update period can be set using the quantized distance.

The classification value generation unit 322 quantizes a distance to at least one point in the point cloud measured based on the sensor using the first coordinate information, the second coordinate information, and the depth information to classify a distance. Produces

는 거리 분류값, T는 행렬의 전치를 의미한다. 분류값 생성부(322)는 포인트 클라우드 내의 임의의 포인트에 센서로부터 측정되는 깊이(거리)에 따라 서로 다른

거리 분류값을 할당한다. 거리 분류값은 구체적으로 하기의 수학식 3과 같이 표현된다.In Equation 2, p _i refers to an i-th random point in the point cloud, d _i is a distance to an arbitrary point based on the sensor,

Is the distance classification, and T is the transpose of the matrix. The classification value generation unit 322 is different depending on the depth (distance) measured from the sensor at any point in the point cloud.

Assign a distance classification value. The distance classification value is specifically expressed by Equation 3 below.

는 임의의 포인트 p_i를 입력으로 하는 거리 분류 함수,

는 센서에서 측정되는 임의의 포인트까지의 깊이(거리)를 입력으로 하는 스텝함수를 이용하여 표현된 거리 분류값을 의미한다. 예를 들어, 분류값 생성부(322)는 스텝 함수를 이용하여 임의의 포인트까지의 거리를 양자화하여 거리 분류 값을 생성할 수 있고, 갱신 주기 설정부(320)는 임의 포인트와 관련된 센서의 자세 정보 및 공간 정보 중 적어도 하나를 입력으로 하고, 스텝 함수를 통하여 생성된 거리 분류값을 출력으로 하는 거리 분류 함수를 이용하여 갱신주기를 설정할 수 있다. 상기 거리 분류 함수의 입력으로 표기된 p_i는 임의의 i번째 포인트를 나타내거나, i번째 포인트와 관련된 센서의 자세 정보 및 공간 정보를 나타낼 수 있다.Where T is the update cycle,

Is a distance classification function with an arbitrary point p _i as input.

Means a distance classification value expressed using a step function that takes as input the depth (distance) to an arbitrary point measured by the sensor. For example, the classification value generation unit 322 may quantize a distance to an arbitrary point using a step function to generate a distance classification value, and the update period setting unit 320 may posture the sensor associated with the arbitrary point An update period may be set using a distance classification function that takes at least one of information and spatial information as an input and outputs a distance classification value generated through a step function. P _i indicated as the input of the distance classification function may indicate an arbitrary i-th point or posture information and spatial information of a sensor associated with the i-th point.

The categorization unit 340 includes a filtering unit 342. For example, the categorization unit 340 categorizes the generated point cloud into at least one group using the different update cycles. For example, the categorization unit 340 categorizes the point clouds generated by the point cloud generation unit 100 into at least one group by using different update cycles set by the update cycle setting unit 320. In the present invention, the update period is a number related to the number of points in at least some regions included in the point cloud required to update the occupancy of the grid cells, and may be expressed by point intensity or point density. For example, the categorization unit 340 categorizes the point clouds into at least one group, and the categorized point clouds can be used for each group when updating the occupancy of grid cells at positions in a corresponding three-dimensional space.

The filtering unit 342 filters points in at least some areas of the categorized point cloud using different update periods for each group of the categorized point cloud. For example, in consideration of the fact that the sparity of the point cloud is different based on the sensor location, the filtering unit 342 generally updates differently set using the distance classification value in the update period setting unit 320. Filter points of at least some areas in the point cloud categorized according to the period. The grid map generating apparatus 10 can filter a grid according to an update cycle set in consideration of a distance classification value even if points in at least some areas in the point cloud are removed, so that a grid map without reducing the quality of the generated grid map can be generated. have.

The grid cell search unit 360 searches for grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud by using sweep lines directed according to the first coordinate information and the second coordinate information. For example, the sweep line used by the grid cell search unit 360 is a tracking ray used in the ray-casting technique, and searches for grid cells generated at the same coordinates as the point cloud is located in the 3D space.

The occupancy update unit 380 calculates the occupancy of the grid cell by using the occupancy strength of points in the categorized point cloud swept on the sweep line, and the grid cell according to the increase or decrease of the logarithm of the calculated occupancy. Update the occupancy of.

Here, m is the occupancy probability of the grid cells (Occupancy Probability) m ₁ ~ m _N is the occupancy probability of the 1 ~ N-th grid cell, T is the transpose of the matrix. In Equation 4, the probability of occupying each grid cell can be normalized to 0 and 1. Occupancy through occupancy probability can be expressed as follows.

는 시간 t에서 i번째 격자 셀의 로그 대수로 표현된 점유도를 의미하고,

는 i번째 격자셀이 점유되었을 확률,

는 i번째 격자셀이 점유되지 않았을 확률을 의미한다. mi는 i번째 격자셀의 점유 확률을 의미하고, z1:t는 시간 t에서 센서에서 측정한 i번 포인트의 거리 및 각도, x1:t는 시간 t에서 측정된 센서의 제1 좌표 정보 및 제2 좌표 정보를 나타낸다. 점유도 갱신부(380)는 격자셀의 점유 확률을 점유도로 변환하고, 변환된 점유도를 증감하여 격자셀의 점유도를 갱신한다. 점유도 갱신부(380)가 격자셀의 점유도와 점유 확률을 변환하는 방법은 다음과 같다.From here

Is the occupancy represented by the logarithmic log of the i-th grid cell at time t,

Is the probability that the i-th grid cell is occupied,

Is the probability that the i-th grid cell is not occupied. mi is the probability of occupancy of the i-th grid cell, z1: t is the distance and angle of point i measured by the sensor at time t, and x1: t is the first coordinate information and second of the sensor measured at time t. Indicates coordinate information. The occupancy update unit 380 converts the occupancy probability of the grid cells to occupancy, and increases or decreases the converted occupancy to update the occupancy of the grid cells. A method for the occupancy update unit 380 to convert the occupancy and occupancy probability of the grid cell is as follows.

Here, l _{i, j, k} is the occupancy represented by the logarithmic logarithm of the grid cell having the coordinates of (I, j, k), m _{i, j, k} is the coordinates of (I, j, k) Indicates the probability of occupancy of the grid cells. The occupancy update unit 380 may convert the occupancy probability of the grid cells to the occupancy expressed in logarithms using Equation (6).

Here, l _{i, j, k} is the occupancy represented by the logarithmic logarithm of the grid cell having the coordinates of (I, j, k), m _{i, j, k} is the coordinates of (I, j, k) Indicates the probability of occupancy of the grid cells. The occupancy update unit 380 may convert the updated occupancy of a grid cell into the occupancy probability of the corresponding grid cell by using Equation (7).

For example, in the present invention, the occupancy probability of the grid cell used by the occupancy update unit 380 may be provided as the occupancy strength of the grid cell, and the occupancy probability and occupancy strength are coordinates in the 3D space of the grid cell. In can mean how many points in the point cloud are swept on the sweep line. The occupancy update unit 380 is a grid corresponding to the coordinates in the three-dimensional space of the swept point using Equation 8 below when there is a point in the point cloud that is swept in the sweep line emitted from the position of the sensor. Update the cell occupancy.

러닝 레이트(learning rate)로 실험적으로 결정되는 가중치이다. 점유도 갱신부(380)는 센서의 위치로부터 출사되는 스윕라인에 스윕되는 포인트 클라우드 내의 포인트가 존재하지 않는 경우 하기의 수학식 9을 이용하여 스윕된 포인트의 3차원 공간상에서의 좌표에 대응되는 격자셀의 점유도를 갱신한다.Here, l _{i, j, k} is the occupancy represented by the logarithmic logarithm of the grid cell with the coordinates of (i, j, k), m _{i, j, k} is the coordinates of (i, j, k) Occupancy probability of grid cells,

It is a weight that is experimentally determined by the learning rate. The occupancy update unit 380 is a grid corresponding to the coordinates in the three-dimensional space of the swept point using Equation 9 below when there is no point in the point cloud that is swept in the sweep line emitted from the position of the sensor. Update the cell occupancy.

는 러닝 레이트(learning rate)로 실험적으로 결정되는 가중치이다. 점유도 갱신부(380)는 러닝 레이트를 결정하여 점유도 갱신 과정의 민감도(Sensitivity)를 조절할 수 있다. 예를 들어, 점유도 갱신부(380)가 러닝 레이트의 값을 높게 결정하는 경우, 적은 양의 포인트 클라우드 내의 포인트에도 민감하게 반응하여 해당 격자셀 내 점유도를 빠르게 갱신할 수 있다. 본 발명의 일 실시 예에 따른 격자셀의 점유도 갱신 과정을 설명하면, 점유도 갱신부(380)는 스윕라인에 스윕되는 포인트 클라우드 내의 포인트가 발견되면 해당 포인트의 3차원 공간상의 좌표에 대응되는 격자셀의 점유확률을 계산하고, 계산된 점유확률을 로그대수를 이용하여 점유도로 변환하고, 상기 수학식 8 또는 9를 이용하여 해당 격자셀의 점유도를 갱신하고, 상기 갱신된 점유도를 다시 수학식 7을 이용하여 점유 확률로 변환하여 해당 격자셀에 점유 확률을 저장할 수 있다.Here, l _{i, j, k} is the occupancy represented by the logarithmic logarithm of the grid cell having the coordinates of (I, j, k), m _{i, j, k} is the coordinates of (I, j, k) Occupancy probability of grid cells,

Is a weight determined experimentally as a learning rate. The occupancy update unit 380 may determine the running rate to adjust the sensitivity of the occupancy update process. For example, when the occupancy update unit 380 determines a high value of the running rate, the occupancy in the grid cell can be quickly updated by sensitively responding to points in a small amount of point clouds. When explaining the process of updating the occupancy of a grid cell according to an embodiment of the present invention, the occupancy update unit 380 corresponds to the coordinates in the 3D space of the point when a point in the point cloud that is swept in the sweep line is found. Calculate the occupancy probability of the grid cell, convert the calculated occupancy probability to occupancy using logarithmic log, update the occupancy of the grid cell using Equation 8 or 9, and re-express the updated occupancy The occupancy probability can be stored in the corresponding grid cell by converting to the occupancy probability using 7.

The integration unit 390 sets and integrates different weights in consideration of the first coordinate information, the second coordinate information, and the depth information in the grid cell in which the occupancy is updated. For example, after the occupancy update unit 380 updates the occupancy of the grid cells, the integration unit 390 may integrate the updated grid cells by setting different weights according to the attitude information and spatial information of the sensor. have.

2 is an enlarged block diagram of the map generation unit 300 in the embodiment of FIG. 1.

The map generation unit 300 includes an update period setting unit 320, a categorization unit 340, a grid cell search unit 360, an occupancy degree updating unit 380, and an integration unit 390. For example, the map generation unit 300 categorizes the generated point cloud according to different update cycles using the sensor's posture information and the spatial information, and the categorized point cloud in the three-dimensional space. The grid map is generated by updating the occupancy of the grid cells generated in consideration of the coordinates with the different update cycles.

In the present invention, the map generating unit 300 may generate different grid maps according to the time when the sensor measures spatial information and the sensor calculates the attitude information, and generates time and attitude information for measuring the spatial information. The 3D grid maps reflecting time information may be generated by integrating the grid maps generated differently according to the calculated time. For example, the update period setting unit 320 uses the distance classification function to classify the distance according to the quantized distance of any point specified by the depth information included in the first coordinate information, the second coordinate information, and the spatial information. You can create, and set the update cycle using the generated distance classification value. The specific method for the update period setting unit 320 to set the update period is the same as described above, and thus is omitted.

The categorization unit 340 categorizes the generated point cloud into at least one group using the different update cycles. The categorization unit 340 categorizes the generated point cloud using an update cycle, and the occupancy update unit 380 updates grid cells corresponding to coordinates in the three-dimensional space of the group of point clouds categorized by the same update cycle. Therefore, the number of points to be used may be provided in the same way. The grid cell search unit 360 searches for grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud by using sweep lines directed according to the first coordinate information and the second coordinate information. The configuration of the sweep line for the grid cell search unit 360 to search for the grid cells is the same as described above, and thus is omitted.

The occupancy update unit 380 generates grid maps by updating grid cells corresponding to the spatial coordinates of the generated point cloud with different update cycles. The process of updating the occupancy of the grid cell by the occupancy update unit 380 is the same as described above, and thus is omitted. The occupancy of the grid cells updated by the occupancy update unit 380 may be leveled between 0 and 1. The integration unit 390 sets and integrates different weights in consideration of the first coordinate information, the second coordinate information, and the depth information in a grid cell having an updated occupancy.

For example, the grid map generating device 10 measures a three-dimensional space according to the position of a specific sensor at time t1 to generate a grid map1, and at time t2 to a different coordinate from the position of the sensor at time t1. A more accurate grid map can be generated by adding grid map 2 generated by measuring a 3D space using a positioned sensor.

3 is an enlarged block diagram of the update period setting unit 320 in the embodiment of FIG. 2.

The update period setting unit 320 includes a classification value generation unit 322. The classification value generation unit 322 quantizes a distance to at least one point in the point cloud measured based on the sensor using the first coordinate information, the second coordinate information, and the depth information by a step function Create distance classification values. The configuration of the distance classification function and the step function used by the classification value generation unit 322 to generate the distance classification value is omitted as described above.

4 is an enlarged block diagram of the categorization unit 340 in the embodiment of FIG. 2.

The categorization unit 340 includes a filtering unit 342. The filtering unit 342 filters points in at least some areas of the categorized point cloud using different update periods for each group of the categorized point cloud. In the present invention, the update period is a concept corresponding to point intensity and density, and may mean the number of points in the point cloud required to update the occupancy of grid cells.

When the grid map generating apparatus 10 of the present invention is used, a grid map can be generated more effectively than a conventional grid map generating technique. As a result of generating a grid map according to an embodiment, when generating a grid map without categorizing point clouds according to different update cycles under the same conditions, it took 0.32 seconds to generate the grid map, but by setting different update cycles When the point cloud is categorized and the occupancy of the grid cells is updated using the categorized point cloud with different update cycles, it may take 0.21 seconds to generate the grid map.

5 is a flowchart of a grid map generating method according to an embodiment of the present invention.

The method of generating a grid map of the present invention includes the following steps performed in time series in a grid map generating apparatus.

In S100, the point cloud generation unit 100 generates different point clouds according to the attitude information of the sensor by using a sensor that measures spatial information representing a 3D space including at least one object. For example, the point cloud generated by the point cloud generating unit 100 may be configured as a three-dimensional space as a point group having color and distance information. The specific method for the point cloud generation unit 100 to generate the point cloud using the RGB image and the depth image is as described above.

In S200, the posture information calculating unit 200 performs image-based SLAM based on the measured spatial information to calculate posture information regarding the position and direction of the sensor. For example, the posture information calculating unit 200 may extract a visual feature point from the RGB image input from the sensor, and calculate 6 degree of freedom posture as posture information of the sensor using 3D SLAM. The detailed method of calculating the attitude information of the sensor by the attitude information calculating unit 200 is the same as described above, and thus is omitted.

In S300, the map generation unit 300 categorizes the generated point cloud according to different update cycles using the sensor's posture information and the spatial information, and coordinates the categorized point cloud in the three-dimensional space. The grid map is generated by updating the occupancy of the grid cells generated in consideration with the different update cycles. The map generator 300 may generate different grid maps according to the time at which the sensor measures the 3D space, and integrates the grid maps generated by measuring the same 3D space at different times to create a new grid map. It can be generated is omitted as described above.

FIG. 6 is an enlarged flow chart of the step of generating a grid map in the embodiment of FIG. 5.

In S320, the update period setting unit 320 based on the posture information of the sensor and the depth information included in the spatial information, points of at least some areas included in the point cloud required to update the occupancy of the grid cell The update cycle for numbers is set as a distance classification function. The detailed method of setting the update period by the update period setting unit 320 is the same as described above, and thus is omitted. In S340, the categorization unit 340 categorizes the generated point cloud into at least one group using different update cycles. The update period used by the categorization unit 340 to categorize the point cloud may be set based on the distance from the sensor to the point in the point cloud, so that at least one group in the point cloud categorized by the categorization unit 340 is categorized. It can be arranged radially relative to the sensor.

In S360, the grid cell search unit 360 searches for grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud using sweep lines directed according to the first coordinate information and the second coordinate information. . The point cloud generated by the point cloud generation unit 100 may be generated differently with time, and grid cells searched corresponding to coordinates in the three-dimensional space of the generated point cloud may also be provided differently.

In S380, the occupancy update unit 380 calculates the occupancy of the grid cell by using the occupancy strength of the points in the categorized point cloud swept in the sweep line, and the calculated occupancy update unit 380 is calculated. The occupancy of the grid cell is updated according to the increase or decrease of the logarithm of the occupancy. The specific method for the occupancy update unit 380 to update the occupancy is the same as described above, and thus is omitted.

7 is a flowchart of a method for generating a grid map according to another embodiment of the present invention.

In S900, the grid map generating device 10 receives an RGB image and a depth image in a 3D space by using a sensor device such as Kinect described above. In S920, the feature point extracting unit and the attitude information calculating unit 200 extracts the feature points from the RGB image, and calculates the 6 degree of freedom of the sensor using 3D SLAM. In S940, the point cloud generator 100 generates a point cloud using an RGB image and a depth image. In S960, the map generation unit 300 updates the DDU (Distance Dependent Update) -based voxel map to generate a voxel map. In S980, the grid map generating device 10 outputs a 3D environment map using the generated voxel map.

8 is an enlarged flow diagram of steps of updating and generating a DDU (Distance Dependent Update) based voxel map in the embodiment of FIG. 7.

In S962, the update period setting unit 320 sets the update period using a distance classification function and a step function. For example, the update period setting unit 320 quantizes a distance from a sensor to a point in a point cloud measured by a sensor using a distance classification function and a step function to generate distance classification values, and according to the generated distance classification values The configuration for setting another update cycle is the same as described above, and thus is omitted.

In S964, the grid cell search unit 360 searches for voxels whose points are hit through ray-tracing. In the present invention, the grid cell includes a voxel as a unit grid having a volume element in a 3D space, and the grid map generated by the grid map generating device 10 may be provided as a voxel map. In S966, the occupancy degree updating unit 380 updates the occupancy degree of the 3D voxel. The specific method for the occupancy update unit 380 to update the occupancy is the same as described above, and thus is omitted.

9 is an exemplary view showing an RGB image measured by a sensor for measuring a three-dimensional space of the present invention.

The sensor for measuring the three-dimensional space included in the grid map generating apparatus 10 of the present invention includes an RGB-D image camera and a TOF depth camera, and may be preferably provided as a Kinect sensor. The grid map generating apparatus 10 may receive an RGB image including a light using a sensor.

10 is an exemplary view showing a depth image of a 3D space measured by a sensor for measuring a 3D space of the present invention.

The grid map generating apparatus 10 of the present invention may include a Kinect or TOF type sensor, and receive a depth image according to depth (distance) information measured from the sensor through the sensor. For example, the grid map generating apparatus 10 may measure the depth information of the three-dimensional space by measuring the time to return by reflecting by emitting light.

11 is an exemplary diagram showing a grid map generated by a grid map generating apparatus based on posture information and spatial information of a sensor measured at a specific time at an arbitrary sensor position.

The 3D voxel map generated by the DDU (Distance Dependent Update) method of the grid map generating apparatus 10 of the present invention appears as shown in FIG. 10. The six-degree-of-freedom attitude of the sensor uses image-based 3D SLAM technology, and the generation of the 3D grid map is based on the above-described distance-dependent update method.

12 is an exemplary diagram illustrating an empty grid map generated by a grid map generating apparatus according to an embodiment.

The grid map generating apparatus 10 of the present invention can direct a point cloud using a sweep line emitted from a position on a sensor, and if there is no point in the point cloud swept from the directed sweep line, 3D of the point It is possible to update the occupancy of the grid cells corresponding to the coordinates in space low. When the occupancy updated by the occupancy update unit 380 is low, the grid cells are empty, and the grid map generated by the grid map generation device 10 may be illustrated as in FIG. 12.

13 is an exemplary view showing a grid map generated by the grid map generating apparatus 10 according to another embodiment.

The grid map generating apparatus 10 of the present invention can update the occupancy of grid cells by setting different update cycles for the number of points in at least some areas in the point cloud corresponding to the grid cells. For example, if the point cloud on the three-dimensional space generated by the point cloud generator 100 is not swept by a sweep line, the occupancy of the grid cells may be empty, and the grid generated by the grid map generator 10 At least a part of the map may appear as an empty space as shown in FIG. 5.

14 is an exemplary view illustrating a grid map generated by the grid map generating apparatus 10 according to another embodiment.

The grid map generating apparatus 10 of the present invention can generate a grid map having 4 m * 8 m * 3 m in width, height, and height, respectively, and can generate a grid map in a short time compared to a conventional map generation technology. As described above

All or part of the method of one embodiment of the present invention described above may be embodied in the form of a computer-executable recording medium, such as a program module, being executed by a computer. Here, the computer-readable medium can be any available medium that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Further, all or part of a method according to an embodiment of the present invention includes instructions executable by a computer and may be embodied as a computer program (or computer program product) recorded on a medium. The computer program includes programmable machine instructions processed by a processor and may be implemented in a high-level programming language, object-oriented programming language, assembly language, or machine language. . In addition, the computer program may be recorded on a tangible computer-readable recording medium (eg, memory, hard disk, magnetic / optical medium, or solid-state drive (SSD), etc.).

Accordingly, a method according to an embodiment of the present invention may be implemented by executing a computer program as described above by a computing device. The computing device may include at least some of a processor, a memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and may be mounted on a common motherboard or mounted in other suitable ways.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains may make various modifications, changes, and substitutions without departing from the essential characteristics of the present invention. will be. Therefore, the embodiments and the accompanying drawings disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain the scope of the technical spirit of the present invention. . The scope of protection of the present invention should be interpreted by the claims below, and all technical spirits within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (20)

A point cloud generator for generating different point clouds according to the posture information of the sensor by using a sensor that measures spatial information representing a three-dimensional space including at least one object;
A posture information calculating unit that calculates posture information regarding the position and direction of the sensor by performing image-based SLAM based on the measured spatial information; And
The generated point cloud is categorized according to different update cycles using the posture information of the sensor and the spatial information, and the occupancy of the generated grid cells is considered in consideration of coordinates in the three-dimensional space of the categorized point cloud. A map generation unit updating the different update cycles to generate a grid map; It includes,
The map generation unit generates the grid map using the calculated attitude information,
The posture information calculator includes a feature point extractor for extracting feature points in consideration of a change in pixel values for each pixel in an image image generated using the spatial information, and calculating posture information of the sensor using the extracted feature points Grid map generating device, characterized in that. delete According to claim 1,
The posture information includes first coordinate information about the position of the sensor and second coordinate information about the direction of the sensor,
The spatial information includes color information of the 3D space and depth information of the 3D space measured based on the sensor. delete The method of claim 3, wherein the map generating unit
An update cycle for the number of points in at least some regions included in the point cloud required to update the occupancy of the grid cell using a distance classification function that inputs at least one of the posture information of the sensor and the spatial information as input. An update cycle setting unit for setting; Further comprising,
A grid map generation device, characterized in that the grid map is generated using a point cloud categorized according to the set update cycle. The method of claim 3, wherein the map generating unit
A categorization unit categorizing the generated point clouds into at least one group using the different update cycles; Further comprising,
A grid map generating apparatus, characterized in that the grid map is generated using the categorized point cloud. The method of claim 3, wherein the map generating unit
A grid cell search unit for searching grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud by using sweep lines directed according to the first coordinate information and the second coordinate information; Further comprising,
A grid map generating apparatus characterized in that the grid map is generated by updating the occupancy of the searched grid cells. The method of claim 6, wherein the categorization unit
A filtering unit that filters points of at least a partial region of the categorized point cloud using different update periods for each group of the categorized point cloud; Further comprising,
A grid map generating apparatus, characterized in that the point cloud is categorized based on the filtered points of at least some areas. The method of claim 7, wherein the map generation unit
Occupancy update to calculate the occupancy of the grid cell using the occupancy strength of points in the categorized point cloud swept on the sweep line, and to update the occupancy of the grid cell according to the increase or decrease of the logarithm of the calculated occupancy part; Further comprising,
A grid map generation device, characterized in that the grid map is generated using the grid cells with the updated occupancy. The method of claim 5, wherein the update cycle setting unit
A classification value that quantizes a distance to at least one point in the point cloud measured based on the sensor using the first coordinate information, the second coordinate information, and the depth information as a step function to generate a distance classification value Generation unit; Further comprising,
A grid map generating apparatus, characterized in that the update period is set using the generated classification value. The method of claim 9, wherein the map generating unit
An integration unit that sets and integrates different weights in consideration of the first coordinate information, the second coordinate information, and the depth information in the grid cell in which the occupancy is updated; Further comprising,
A grid map generating apparatus characterized in that the grid map is generated using the grid cells integrated with the different weights. Generating different point clouds according to the posture information of the sensor using a sensor measuring spatial information representing a 3D space including at least one object;
Calculating posture information regarding the position and direction of the sensor by performing image-based SLAM based on the measured spatial information; And
The generated point cloud is categorized according to different update cycles using the posture information of the sensor and the spatial information, and the occupancy of the generated grid cells is considered in consideration of coordinates in the three-dimensional space of the categorized point cloud. Generating a grid map by updating with the different update cycles; It includes,
In the generating of the grid map, the grid map is generated using the calculated posture information,
The calculating of the posture information may include extracting feature points in consideration of a change in pixel values for each pixel in an image image generated using the spatial information, and calculating posture information of the sensor using the extracted feature points How to generate a grid map. The method of claim 12,
The posture information includes first coordinate information about the position of the sensor and second coordinate information about the direction of the sensor,
The spatial information includes color information of the 3D space and depth information of the 3D space measured based on the sensor. The method of claim 13, wherein the step of generating the grid map
Setting an update cycle regarding the number of points in at least some areas included in the point cloud required to update the occupancy of the grid cell based on the posture information of the sensor and the spatial information; Further comprising,
A grid map generation method characterized by generating the grid map using a point cloud categorized according to the set update cycle. The method of claim 13, wherein the step of generating the grid map
Categorizing the generated point clouds into at least one group using the different update cycles; Further comprising,
A grid map generation method characterized by generating the grid map using the categorized point cloud. The method of claim 13, wherein the step of generating the grid map
Searching for grid cells corresponding to coordinates in the three-dimensional space of the categorized point cloud by using sweep lines directed according to the first coordinate information and the second coordinate information; Further comprising,
A grid map generation method characterized in that the grid map is generated by updating the occupancy of the searched grid cells. 16. The method of claim 15, wherein the categorizing step
Filtering points of at least some regions of the categorized point cloud using different update periods for each group of the categorized point cloud; Further comprising,
A method of generating a grid map, wherein the point cloud is categorized based on the filtered points of at least some areas. The method of claim 16, wherein the step of generating the grid map
Calculating the occupancy of the grid cell by using the occupancy strength of points in the categorized point cloud swept on the sweep line, and updating the occupancy of the grid cell according to the increase or decrease of the logarithm of the calculated occupancy; Further comprising,
A grid map generation method, characterized in that the grid map is generated using the grid cells with the updated occupancy. 15. The method of claim 14, The step of setting the update cycle
Generating a distance classification value by quantizing a distance to at least one point in the point cloud measured based on the sensor using the first coordinate information, the second coordinate information, and the depth information; Further comprising,
A grid map generation method characterized by setting the update period using the generated classification value. A program stored in a computer-readable recording medium for realizing the grid map generating method according to any one of claims 14 to 19 through being executed by a processor.

Images