KR101864127B1 - Apparatus and method for environment mapping of an unmanned vehicle - Google Patents

Abstract

A method for mapping an environment around an unmanned vehicle, the method comprising: extracting a superpixel from any one of images sensed by at least one camera provided in the unmanned vehicle according to a predetermined method; Extracting depth data and 3D point data from a result of sensing the periphery of the unmanned vehicle in a three-dimensional scanner provided in the stereoscopic image display apparatus, and outputting the depth data and 3D point data to the extracted super pixels using a predetermined projection matrix And superimposing a super pixel index at each position of the superpixel; clustering each grid constituting the superpixel according to the assigned superpixel index; Based on the detected obstacle, the activated state, the inactivated state, and the activated state or inactive state Estimating a dynamic state of each lattice community at a time point k + 1, which is the next time point of the current time k, based on the traveling speed and the rotation angle of the unmanned vehicle; Detecting particles moving between the lattice clusters based on the predicted dynamic state and estimating each lattice cluster state at the next time point according to the detected particles, And updating the state of each lattice community by combining the states of the lattice communities measured at the current point k.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and an apparatus for mapping an environment for an unmanned vehicle,

The present invention relates to an apparatus and method for mapping an environment in an unmanned vehicle, and for tracking and predicting detected obstacles more efficiently.

For an autonomous navigation system of an unmanned vehicle, it is necessary to measure the surrounding environment through a sensor mounted on the vehicle, and to estimate a path by estimating dynamic and static obstacle movement. In recent years, Occupancy Grid Filtering (OGF), which maps the surrounding environment to discrete and uniform lattice units so as to enable real-time calculation for the motion estimation and prediction of the obstacle in the unmanned vehicle, Has been proposed. The OGF includes a process of predicting a position of an obstacle to be inputted next based on a current lattice state, i.e., a state in which a surrounding environment is mapped to a lattice. On the other hand, the path prediction in a specific environment involves the representation of a static object and the estimation of the motion state of the obstacle by tracking the dynamic object.

This OGF is an important technology for routing autonomous vehicles in a dynamic environment. This method expresses the surrounding environment as a discrete lattice environment and can easily represent the activation state of each lattice stochastically. Although OGF can not be used for high-dimensional tasks such as object recognition or semantic recognition around an autonomous vehicle, it can be easily applied to tracking of dynamic objects or static objects, positional recognition of tracked objects, and path prediction. have.

Recently, instead of performing tasks using a single sensor such as a monocular or binocular camera, a laser scanner, or a radar, it utilizes the mutual complementarity of multiple sensor modalities for various environmentally adaptive sensing. And, when using multiple sensors, multiple grids activated by one obstacle have a similar dynamic state.

It is possible that different prediction functions may be applied, even though they have the same dynamic state, if they handle such a group of grids independently. These problems continue to accumulate errors over time. Also, processing each of the grid clusters activated by one obstacle each time reduces the efficiency of computation.

Accordingly, there is a problem that it is difficult to apply the method of estimating the path of the dynamic and static obstacle by estimating the motion of the surrounding environment by using the existing single sensor.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for enabling an unmanned vehicle to more effectively measure an environment and track and predict a detected obstacle when multiple sensors are used.

Another object of the present invention is to provide an apparatus and method for more accurately tracking and predicting obstacles by complementing each other's disadvantages among a plurality of sensors having different functions.

According to another aspect of the present invention, there is provided a method of mapping an environment around an unmanned vehicle, the method comprising: Extracting depth data and 3D point data from a result of sensing the periphery of the unmanned vehicle by a three-dimensional scanner provided in the unmanned vehicle, and extracting depth data and 3D point data using a predetermined projection matrix A step of assigning a super pixel index to each position of the super pixel by projecting the depth data and the 3D point data, clustering each grid constituting the super pixel according to the assigned super pixel index, Each lattice cluster of pixels may be classified into an active state, a deactivated state, And determining a dynamic state of each grid cluster at a time point k + 1, which is the next time point of the current time k, based on the movement speed and the rotation angle of the unmanned vehicle, The method includes the steps of: predicting particles moving in each grid cluster based on a predicted dynamic state, and estimating each grid cluster state at the next point of time according to the detected particles, And updating the state of each grid cluster by combining the grid cluster state predicted at time k-1 and the state of each grid cluster measured at the current time k.

In one embodiment, the clustering of the respective gratings is a step of clustering three-dimensional points having the same super pixel index and depth information.

According to an embodiment of the present invention, the step of classifying the state of each lattice community may include a step of classifying the states of the lattice communities according to the activated particles and the deactivated particles among the particles constituting one of the lattice communities, A second step of calculating a potential region based on the active region and the inactive region divided in the one of the plurality of lattice communities; and a second step of calculating an inactive mass from the inactive region, an inactive mass from the inactive region, and A third step of calculating a potential mass from the potential region, a fourth step of classifying the states of any one of the grid clusters based on the calculated activation mass, inactivation mass, and potential mass, The first to fourth steps are repeated for other lattice communities other than the lattice community, Characterized in that it comprises the step that distinguishes the status of each grid clusters that make up the superpixel group.

In one embodiment, the step of predicting the dynamic state of each lattice cluster comprises the steps of: acquiring the moving speed and the rotational angle of the unmanned vehicle based on the inertial sensor provided in the unmanned vehicle; Calculating a rotation angle and an operation distance of the unmanned vehicle during a predetermined period of time based on the rotation angle and the rotation angle of the vehicle; calculating a position source of each grid cluster based on the calculated rotation angle and the operation distance for the predetermined time; And calculating a position of each of the lattice communities based on the calculated positional information of the lattice constellations and the predetermined discrete Gaussian noise, And predicting the dynamic state.

According to an aspect of the present invention, there is provided an apparatus for mapping an environment around an unmanned vehicle, the apparatus including a three-dimensional scanner, at least one camera, and an inertial sensor for sensing a speed and a rotation angle of the unmanned vehicle And superimposed on each region of the super pixels extracted from the image sensed from any one of the at least one cameras based on 3D point data and depth information sensed from the 3D scanner and a predetermined projection matrix, Assigning a pixel index, grouping each grid constituting the superpixel based on the assigned superpixel index, recognizing an obstacle around the unmanned vehicle based on the superpixel index, Each grid of superpixels is activated and deactivated, Estimating a dynamic state of each lattice cluster at a point in time after the current point k on the basis of the moving speed and the rotation angle of the unmanned vehicle, calculating a state of each lattice cluster measured at the current point k, and a controller for combining the states of the respective lattice communities predicted at the time k-1 to update the state of each lattice community.

In one embodiment, the controller performs clustering between three-dimensional points and depth information having the same super pixel index to group clusters constituting the super pixels.

In one embodiment, the controller divides each of the grid groups into an activation region, a deactivation region, and a potential region based on the activated particles and the deactivated particles, among the particles constituting the respective grid clusters, The active mass, inactivation mass, and potential mass are calculated from the respective lattice communities based on the activation regions, inactive regions, and potential regions of the respective lattice communities, and the activation mass, inactivation mass, and potential mass The state of each grid cluster is divided into the active state, the inactive state, and the potential state.

In one embodiment, the control unit may calculate the rotation angle and the operation distance of the unmanned vehicle during a predetermined time based on the obtained speed and rotation angle, and calculate the rotation angle and the operation distance based on the calculated rotation angle and the operation distance, Calculating a location source of the lattice community based on the location of each of the lattice communities based on the location sources of the respective lattice communities and calculating a location of each of the lattice communities based on the positions of the compensated lattice communities and the next discrete Gaussian noise, And estimating a dynamic state of each lattice cluster of the plurality of lattice constellations.

A method and an apparatus for mapping an environment of an unmanned vehicle according to the present invention will now be described.

According to at least one of the embodiments of the present invention, the OGF process proceeds more efficiently through the lattice clustering, thereby shortening the time required for existing obstacle tracking and prediction.

According to at least one of the embodiments of the present invention, the same prediction function can be applied to the grids activated by one obstacle, thereby making it possible to lower the error occurrence rate in the tracking and prediction of the obstacle have.

1 is a block diagram for explaining a configuration of an environment mapping apparatus according to an embodiment of the present invention.
2A and 2B illustrate an example of an unmanned vehicle having an environment mapping apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating an operational flow of the environment mapping apparatus according to an embodiment of the present invention.
4 is a conceptual diagram illustrating an example of assigning an index of a superpixel on three-dimensional data from a superpixel space of a monocular image in an environment mapping apparatus according to an embodiment of the present invention.

It is noted that the technical terms used herein are used only to describe specific embodiments and are not intended to limit the invention. Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In this specification, " comprises " Or " include. &Quot; Should not be construed to encompass the various components or steps described in the specification, and some of the components or portions may not be included, or may include additional components or steps And the like.

Further, in the description of the technology disclosed in this specification, a detailed description of related arts will be omitted if it is determined that the gist of the technology disclosed in this specification may be obscured.

In order to facilitate understanding of the present invention, the present invention briefly describes clustering of gratings by applying the superpixel concept to a grating environment. Superpixels are medium-level visual clues, used in various computer vision algorithms such as object segmentation, recognition, and tracking. The present invention performs clustering between three-dimensional point and depth information having the same super pixel index by projecting the data of the 3D laser scanner and the depth information map extracted from the camera in the super pixel space of the camera data. This method is more efficient and effective because it applies an equal prediction function to the grid clusters activated by one obstacle.

The present invention also provides a method of representing a state of a lattice in a state in which clustered lattices can be both activated and deactivated, as well as being activated or deactivated, predicting the posterior state of each lattice community through the lattices represented by each state, The state of each grid can be updated based on the predicted result and the state of each grid represented.

FIG. 1 is a block diagram for explaining a configuration of an environment mapping apparatus 100 according to an embodiment of the present invention.

1, the apparatus 100 for mapping the environment according to an exemplary embodiment of the present invention includes a controller 110, a sensor unit 120 connected to the controller 110, and a predictor 130 . The components shown in FIG. 1 are not essential for implementing the environment mapping and obstacle prediction apparatus 100 according to the embodiment of the present invention, and may have more or fewer components than the components listed above have.

First, the sensor unit 120 may include at least one sensor for sensing the environment around the unmanned vehicle. For example, the sensor unit 120 may include a laser scanner 122 and a binocular camera 124 capable of sensing a left eye image and a right eye image. The laser scanner 122 may be a three-dimensional multi-layer laser scanner (LIDAR) capable of scanning the environment around the unmanned vehicle in three dimensions.

2A and 2B show an example of the unmanned vehicle 200 having the laser scanner 122 and the binocular camera 124 as described above.

2A, an unmanned vehicle 200 having an environment mapping apparatus 100 according to an embodiment of the present invention may include a laser scanner 122 and a binocular camera 124. In this case, the laser scanner 122 and the binocular camera 124 may be arranged as shown in FIG. 2B.

On the other hand, the laser scanner 122 can perform accurate data measurements on a unique measurable area, but as can be seen in the laser scanner detection area 220 of FIG. 2B, the detectable area is detected by the image sensor, As shown in FIG. On the other hand, as shown in the left eye image area 210L and the right eye image area 220L of FIG. 2B, the binocular camera 124 is wider than the area where the measurable area is detected through the laser scanner 124, It is difficult to accurately measure the depth information (depth information) as compared with the case of the second embodiment.

In the following description of the present invention, two sensors having different functions are provided to complement each other. The following method of the present invention can easily be applied to other sensor modalities, but it is assumed that extracting monocular image data is a method of extracting super pixels in a monocular image. Accordingly, it is needless to say that the camera 124 included in the sensor unit 120 may be a monocular camera having only one camera. In addition, the sensor unit 120 may further include an inertial sensor (e.g., an Inertial Measurement Unit (IMU) Sensor) to accurately predict the dynamic state of the obstacle.

The sensor unit 120 may sense the environment around the unmanned vehicle 200 through the laser scanner 122 and the binocular camera 124 according to the control of the controller 110. [ And the sensed result can be applied to the control unit 110.

Then, the control unit 110 may extract the super pixels using the ambient environment detection result from the sensor unit 120. [ As described above, the super pixel is used in various vision algorithms such as object segmentation, recognition, and tracking. The super pixel is extracted from any one of the binocular cameras 124 through SLIC (Simple Linear Iterative Clustering) . The control unit 110 converts the depth data and the 3D point data according to the result of the scan from the laser scanner 122 into the extracted 2D image, that is, super pixels extracted from any one of the images using a preset projection matrix So that a super pixel index can be assigned to each position. Here, the superpixel may refer to a form in which pixels having similar information are clustered based on information such as a CIE Lab color space on a 2D image. Each superpixel has a cluster of smaller units than the object division method, and each cluster can mean one superpixel. And each pixel cluster in a 2D image can have its index.

If the super-pixel index is assigned to each position of the super-pixel, the controller 110 may perform clustering between the three-dimensional point and the depth information having the same super-pixel index. And the cluster state of the grid of each superpixel clustered can be displayed.

For example, the control unit 110 may express a cluster state of each grid by an active state, a deactivated state, or a state that can be either an active state or an inactive state (hereinafter, referred to as a potential state).

Here, the active state may mean a state measured by an obstacle above a certain threshold value among the measured values of the sensor that can be located in one grid cluster. For example, assuming that the LIDAR sensor has 10 laser beam scans that can be reflected in any lattice cluster and that the activation threshold is 5, if 6 laser beams are measured against an obstacle, It can be judged that the lattice cluster has become active. And the deactivated state may mean a state measured by an obstacle below a predetermined threshold value among the sensor values that can be located in one lattice cluster. That is, contrary to the above example, if only four beams among the ten measurable laser beam scans are measured against an obstacle, the lattice cluster may be determined as inactive.

Accordingly, the controller 110 can recognize the objects (the lattice communities) constituting the surrounding environment in different states according to the detection result of the surrounding environment inputted from the sensor unit 120. An obstacle around the current unmanned vehicle 200 can be recognized according to the state of the respective grids.

On the other hand, the predictor 130 can predict the cluster state of the next time (k + 1) from the state of each grid cluster under the control of the controller 110. [ That is, the controller 110 may input each grid cluster represented by the active state, the inactive state, and the latent state to the predictor 130 as described above. Then, the prediction unit 130 can predict the dynamic state of the obstacle for a predetermined time based on the moving state of the unmanned vehicle 200, that is, the speed and the rotation angle . Here, the obstacle may refer to each grid cluster. That is, the predicting unit 130 can estimate the movement of each grid cluster based on the dynamic state of each of the input grid clusters. The speed and the rotation angle of the unmanned vehicle 200 can be obtained through the inertial sensor 126 provided in the sensor unit 120. [

Meanwhile, when the dynamic state of the obstacle at time k + 1 is predicted through the predictor 130, the controller 110 can receive the predicted result of the dynamic state of the obstacle from the predictor 130. Then, the controller 110 may combine the state of the grid cluster predicted by the predictor 130 and the grid cluster measured at the actual time k + 1. And the state of the lattice cluster can be updated according to the result of the combination. And to predict a potential obstacle due to the movement of the unmanned vehicle 200 based on the updated grid clustering state.

3 is a flowchart illustrating an operation flow of the environment mapping apparatus 100 according to the embodiment of the present invention.

First, the controller 110 of the environment mapping apparatus 100 according to an exemplary embodiment of the present invention extracts a super pixel based on an image sensed by any one of the binocular cameras 124 as described above S300). The extracted super pixels can be represented by a plurality of grids.

On the other hand, when the super-pixel is extracted in step S300, the controller 110 may project the 3D point data and the depth data sensed by the laser scanner 122 using the predetermined projection matrix P to the extracted super-pixel. Then, a super pixel index related to each position of the super pixel may be allocated (S302).

FIG. 4 illustrates an example in which an index of a superpixel is allocated on three-dimensional data from a superpixel space of a monocular image in the apparatus 100 for mapping an environment according to an embodiment of the present invention.

In FIG. 4 (a), the red boundary shows the boundary of the superpixel, and the set of points imposed on the neighboring vehicle is obtained by dividing the set of three-dimensional points measured from the laser scanner 122 by a 2-dimensional projection matrix Dimensional image can be displayed. (A) Each set of points projected onto a two-dimensional image in the image can be assigned a super-pixel index in the image of the corresponding position, and (b) the image shows the super-pixel index assigned to each point data. (C) The image is an enlarged view of the rear view of an obstacle (eg another vehicle) located in front of an unmanned vehicle in the image (b). After mapping the data measured by the three-dimensional sensor on the two-dimensional grid map, (B) the index of each grid is determined through the super pixel index of each point obtained through the image. At this time, the grids assigned to the same index can be judged to be a lattice cluster having the same dynamic state as (D) image and (D) image.

Equation (1) shows that a super pixel index is assigned according to the predetermined projection matrix P as described above.

는 3D 포인트 데이터 혹은 깊이 영상 데이터의 픽셀을 의미하며, P는 3차원 포인트 데이터를 2차원 데이터 공간, 즉 2차원 영상에 투영하는 투영행렬을 의미한다. sp'와 sp는 각각 3D 포인트 데이터의 슈퍼픽셀 인덱스와 2D 데이터(2차원 영상)의 슈퍼 픽셀 인덱스를 의미할 수 있다. here

Means a pixel of 3D point data or depth image data, and P means a projection matrix for projecting three-dimensional point data into a two-dimensional data space, i.e., a two-dimensional image. sp 'and sp can respectively mean a super pixel index of the 3D point data and a super pixel index of the 2D data (2D image).

If the super-pixel index is assigned to each position of the super-pixel, the controller 110 may perform clustering between the three-dimensional point and the depth information having the same super-pixel index (S304). Here, the lattice clustering is intended to increase the cost and accuracy of computation by assigning the same dynamic state to each of the sets of lattices activated by the same obstacle rather than through different dynamic states.

Meanwhile, when the grid clustering is completed in step S304, the controller 110 can represent each grid cluster in different states (S306). In step S306, the controller 110 may represent the active state or the inactive state according to the ratio of the number of particles activated in each grid cluster.

(예 : 군집 j영역에서 가능한 모든 레이저 스캐너의 레이저 반사 수)와 활성화된 모든 파티클의 수

의 비율로 측정하며, 하기 수학식 2와 같다. 여기서 파티클이란 센서로부터 측정된 3차원 데이터를 의미하며, 활성화된 파티클은 장애물에 부딪힌 3차원 데이터를 의미한다. 예를 들어, 레이저 스캐너(122)의 경우 장애물에 의해 반사된 레이저 빔 하나가 하나의 활성화된 파티클을 의미할 수 있다. Here, the activation probability of each lattice cluster is the number of all possible particles in the lattice cluster j

(Eg, the number of laser reflections of all possible laser scanners in cluster j area) and the number of all active particles

And is expressed by the following equation (2). Here, a particle means three-dimensional data measured from a sensor, and an activated particle means three-dimensional data impinging on an obstacle. For example, in the case of the laser scanner 122, one laser beam reflected by the obstacle may mean one activated particle.

, N은 격자 군집 j에 활성화된 파티클의 수를 의미하며,

는 각 격자 군집 j에 속한 격자 멤버 c의 가능한 모든 파티클의 수를 의미한다.

는 j번째 군집에 속한 i번째 격자를 의미한다. 격자 군집 j가 비활성화 될 확률은

로 표현될 수 있다.Where j is the sum of all grid clusters in the lattice space

, N means the number of particles activated in the lattice cluster j,

Is the number of all possible particles of the grid member c belonging to each grid cluster j.

Is an i-th grid belonging to the j-th cluster. The probability of lattice cluster j being deactivated

. ≪ / RTI >

로 주변 환경의 상태를 확률

로 표현할 수 있다. On the other hand, the controller 110 calculates the two-dimensional grid C = [X, Z] from the data of the laser scanner 122 measured from the time '1' to the current time 'k' Community of

The probability of the state of the surrounding environment

.

는 2차원 위치와 속도 정보를 포함하는 동적 상태

,

와, 활성화 상태 확률

,

, 그리고 각 격자 군집에 속하는 격자 구성원

로 구성된다. 따라서 격자 군집 i의 상태

는 하기 수학식 3과 같이 표현될 수 있다. The state of each lattice cluster

A dynamic state including two-dimensional position and velocity information

,

And an activation status probability

,

, And a lattice member belonging to each lattice community

. Thus, the state of the grid cluster i

Can be expressed by the following equation (3).

는

로 표현될 수 있다. 여기서

는 격자 군집 j의 구성원,

는 격자 군집 j의 활성화 상태 확률(예 : 활성화 또는 비활성화 상태 확률),

는 격자 군집 j의 동적 상태를 의미할 수 있다.Thus, the state of the grid cluster j at time k

The

. ≪ / RTI > here

Is a member of lattice cluster j,

(Eg, activation or deactivation probability) of the grid cluster j,

May denote the dynamic state of the lattice cluster j.

However, most of the gratings in the lattice space do not have information that can be judged as active or inactive. For example, it is difficult to judge the presence of an obstacle by the measurement of the sensor only at the rear part of the grating activated by one obstacle (from the area lying on the straight line of the autonomous vehicle) . However, in real-world environments, these areas have both potential (both probable probabilities) to be activated or deactivated, and this requires more accurate path prediction.

In order to solve this problem, in the present invention, a known area (area where the first obstacle is measured from the sensor's original position) and an unknown area (area that can not be measured after the first obstacle) Shafer's theory of evidence will be used separately.

, 비활성화 질량

, 그리고 알려지지 않은 영역(활성화 상태 또는 비활성화 상태가 될 수 있는 격자 군집의 영역), 즉 잠재적 영역의 질량

이다.

는

로 계산된다. For this, the activation state of each lattice cluster is expressed as a mass function by applying Basic Belief Assignment (BBA). Here, the mass function using BBA is a unit that is used in a system that deals with reliability, by expressing the probability of possibility for all states as a mass function instead of expressing the existing specific state as a probability. That is, for example, when the activation probability is 0.8, the inactivation probability is not represented as 0.2 = 1-0.8, but the states of the activation probability and the inactivation probability are treated independently, and at the same time, Uncertain state) is also expressed as a mass function. At this time, the components of the mass function are the active mass

, Inactive mass

, And the unknown area (the area of the lattice cluster that can be in the active state or the inactive state), that is, the mass of the potential area

to be.

The

.

는

로 표현(representation)될 수 있다. 여기서 상기

는 질량 함수, 즉 활성화 질량

, 비활성화 질량

, 그리고 잠재적 질량

가 반영된 격자 군집의 상태 확률을 의미할 수 있다. As a result, the state of the grid cluster j at time k

The

As shown in FIG. Here,

Is a mass function, that is,

, Inactive mass

, And potential mass

Can be defined as the probability of the state of the grid clusters.

The activation state is defined as the case where most of the particles belonging to the lattice cluster (above the threshold value) collide with the obstacle, and the inactivation is the lattice community lying on the straight line from the unmanned vehicle to the activated lattice The probability of inactivation is high because there is no measurement), and the unknown area may include the area behind the grid clusters determined to be active from the unmanned vehicle.

All grid clusters have all three probabilities (activation, deactivation, potential), and consequently, the initial angular states (activation, deactivation, potential) are as follows.

는 위에서 명시된 확률 측정 방법으로 표현될 수 있다(0~1 의 값)(상기 수학식 2). 그리고

는 무인 차량 ~ 활성화 확률이 0 인 격자 군집에서는 1 로, 그리고

가 0 이상 1 미만인 격자 군집에서는 1 -

으로 표현될 수 있다.

는

로 측정이 될 수 있다. 즉

와

가 없는 영역, 센서의 측정이 아직 이루어지지 않은 영역(

= 0,

= 0)은, 초기 상태로

= 1 이 할당될 수 있다.

Can be expressed by the above-described probability measurement method (a value of 0 to 1) (Equation 2). And

Is 1 for unmanned vehicles ~ lattice communities with 0 activation probability, and

Is 0 or more but less than 1,

. ≪ / RTI >

The

. ≪ / RTI > In other words

Wow

The area where the sensor has not yet been measured (

= 0,

= 0), the initial state

= 1 can be assigned.

상태로부터 다음 시간 k+1 에서의 상태를 예측할 수 있다(S308). 이 모델에서, 본 발명은 각 격자 군집의 사후 상태(Posterior state)를 하기 수학식 4와 같이 할당할 수 있다. If the state of each lattice cluster is expressed as an active state, an inactive state, and a latent state, the controller 110 determines

The state at the next time k + 1 can be predicted (S308). In this model, the present invention can allocate the posterior state of each grid cluster as shown in Equation (4).

를 센서부(120)로부터 획득할 수 있다. 그리고 획득된 정보는 현재 프레임의 시간변화 T동안의 장애물 동적 상태를 정확히 예측하기 위해 사용될 수 있다. 시간 k와 k+1사이의 변화 T동안의 회전각은

로 계산되며, 운전 거리는

로 계산될 수 있다. 그러면 제어부(110)는 이를 이용하여 격자 군집의 위치 근원(origin)을 하기 수학식 5와 같이 계산할 수 있다. For this, the control unit 110 firstly calculates the velocity v and the yaw rate of the unmanned vehicle 200,

Can be acquired from the sensor unit 120. And the obtained information can be used to accurately predict the dynamic state of the obstacle during the time variation T of the current frame. The angle of rotation during the change T between time k and k + 1 is

, And the driving distance

Lt; / RTI > Then, the control unit 110 can calculate the origin of the grid cluster according to Equation (5).

와

는 각각 격자 군집의 x축 크기와 z축 크기를 의미한다. 결과적으로, 각 격자 군집의 위치는 하기 수학식 6과 같이 보상(compensate)될 수 있다. here

Wow

Denote the x-axis size and the z-axis size of the lattice cluster, respectively. As a result, the position of each lattice cluster can be compensated as shown in Equation (6) below.

The controller 110 can predict the dynamic state at k + 1 by applying Equation (7) based on the position information of each lattice cluster in the compensated k.

는 임의적으로 그려진 이산 가우시안 잡음으로 평균값 0과 칼만 필터의 공분산 행렬 Q를 갖는다. here

Is an arbitrarily drawn discrete Gaussian noise with mean value 0 and a covariance matrix Q of Kalman filter.

On the other hand, based on the predicted dynamic state of each lattice cluster, it is possible to calculate the transition state of the particle moving from the cluster a to j during the time change T by the following equation (8).

Where N can be the number of particles moving from grid a to j for k + 1 at time k.

On the other hand, the state of the grid cluster at time k can be made active and dynamic. And, as shown in the mathematical expression 8, the position of the particle belonging to the lattice community at time k + 1 can be predicted based on the dynamic state of the lattice community at time k. Then, the activation state of the corresponding lattice community at time k + 1 can be predicted through the position of the particle at the predicted time k + 1.

도 고려할 수 있다. 본 발명은 이처럼 상기 잠재적 영역의 질량(

,

=

)에서 보이고 있는 바와 같이, 잠재적 영역에 대한 고려를 위해 비활성화 확률 또한 측정하기 위해 시간 종속 인자(time-dependent factor)를 더 이용해 비활성화 상태 예측을 하기 수학식 9와 같이 수행할 수 있다. At this time, particles that have not moved for a time variation T, i.e.,

Can be considered. The present invention thus provides a method for determining the mass of the potential region

,

=

), As shown in Equation (9), the deactivation state prediction can be further performed using a time-dependent factor to measure the deactivation probability for consideration of the potential area.

는, 시간에 따른 불확실성의 증가를 모델링한 시간-절감 인자(time-discounting factor)를 의미할 수 있다. 만약 격자 군집이 일정 시간 동안 갱신 되지 않았을 때, 해당 군집의 기초 신뢰 할당(Basic Belief Assignment, BBA)은 신뢰성이 감소하는 사실을 모델링 한 것이다. here

May refer to a time-discounting factor modeling an increase in uncertainty over time. The basic belief assignment (BBA) of the cluster is modeled when the lattice cluster is not updated for a certain period of time.

If the prediction of the grid cluster state is completed in step S308, the controller 110 may update the state of the grid cluster based on the predicted state k + 1 of the grid cluster S310. In step S310, the state is updated through the grid cluster state at time k + 1 predicted based on the state of the grid cluster measured at time k and the state of the grid cluster measured at time k + 1 . That is, the state of the lattice cluster can be updated through the state of k + 1 predicted at time k and the state measured at actual time k + 1.

을 하기 수학식 10과 같이 얻을 수 있다.This process can be performed through the Dempster-Shafer coupling theory. In step S310, the controller 110 combines the currently observed data (the grid cluster state at time k + 1) and the predicted state at time k (the grid cluster state at the predicted time k + 1). Through this process, post-activation BBA

Can be obtained as shown in Equation (10).

Meanwhile, the controller 110 may perform a pignistic conversion on the result of Equation (10). Here Pigonist transformation is used for decision making. Pigonistic transformation is defined by a linearity requirement and is a way of expressing a state through a linear combination of detailed probabilities for a given state. The results of the pignic transformation are the same as those obtained by applying the pignic transformation to the observed data and the predicted data, respectively.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Accordingly, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the essential characteristics of the invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the scope of the present invention but to limit the scope of the technical idea of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Unmanned vehicle environment mapping device
110: control unit 120: sensor unit
122: laser scanner 124: binocular camera
126: inertia sensor 130:

Claims (8)

A method for mapping an environment in an unmanned vehicle,
Extracting a super pixel from any one of images sensed by at least one camera provided in the unmanned vehicle according to a predetermined method;
Extracting depth data and 3D point data from a result of sensing the periphery of the unmanned vehicle in a three-dimensional scanner provided in the unmanned vehicle;
Projecting the depth data and 3D point data to the extracted super pixels using a predetermined projection matrix and assigning super pixel indices to respective positions of the super pixels;
Clustering each grid constituting the superpixel according to the assigned superpixel index;
Dividing an activation state of each lattice community of the super pixel into an activated state, an inactive state, and a latent state that is not classified as the activated state or the inactivated state based on the recognized obstacle;
Estimating a dynamic state of each lattice community at a time point k + 1 that is the next time point of the current time point k based on the moving speed and the rotation angle of the unmanned vehicle;
Detecting particles moving between the lattice clusters based on the predicted dynamic state, and predicting the activation state of each lattice cluster at the next time point according to the detected particles; And
And updating the activation state of each lattice community by combining the activation state of the lattice community predicted at the k-1 time point before the current point k and the activation state of each lattice community measured at the current point k, Wherein the method comprises the steps of: The method of claim 1, wherein clustering each grid comprises:
Wherein the step of clustering between three-dimensional points and depth information having the same super pixel index is performed. The method according to claim 1, wherein the dividing of the activation states of the grid groups comprises:
A first step of dividing the particles into active areas and inactive areas according to activated particles and deactivated particles among the particles constituting one of the grid clusters;
A second step of calculating a potential area based on the active area and the inactive area divided into one of the grid groups;
A third step of calculating an activation mass from the activation region, an inactive mass from the inactive region, and a potential mass from the potential region;
A fourth step of distinguishing activation states of any one of the grid clusters based on the calculated activation mass, inactivation mass, and potential mass;
And repeating the first to fourth steps in a different lattice community other than any one of the lattice communities described above to distinguish activation states of the lattice communities constituting the superpixel Of the environment. 2. The method of claim 1, wherein the step of estimating the dynamic state of each lattice-
Acquiring a moving speed and an angle of rotation of the unmanned vehicle based on an inertial sensor provided in the unmanned vehicle;
Calculating a rotation angle and an operation distance of the unmanned vehicle during a predetermined time based on the obtained movement speed and rotation angle;
Calculating a location source of each lattice community based on the calculated rotation angle and operation distance for the predetermined time;
Compensating the position of each grid cluster based on the calculated position source; And
Estimating a dynamic state of each grid cluster at the next time point based on the calculated positional information of each grid cluster and predetermined discrete Gaussian noise. An apparatus for performing an environment mapping in an unmanned vehicle,
A sensor unit including a three-dimensional scanner, at least one camera, and an inertial sensor for sensing a speed and a rotation angle of the unmanned vehicle;
A super pixel index is assigned to each area of the super pixel extracted from the image sensed from any one of the at least one camera based on 3D point data and depth information sensed from the 3D scanner and a predetermined projection matrix , Clustering each grid constituting the superpixel based on the assigned superpixel index,
Recognizes an obstacle around the unmanned vehicle based on the super pixel index, and divides an activation state of each grid cluster of the super pixel into an activated state, an inactive state, and a potential state based on the recognized obstacle,
Estimating a dynamic state of each lattice cluster at a point in time after the current point k based on the moving speed and the rotation angle of the unmanned vehicle, calculating a state of each lattice cluster measured at the current point k, And a control unit for updating the activation state of each lattice community by combining the activation states of the respective lattice communities predicted at the one point in time. 6. The apparatus of claim 5,
And clustering the three-dimensional points and the depth information having the same super pixel index to cluster the respective gratings constituting the super pixel. 6. The apparatus of claim 5,
Wherein each of the lattice communities is divided into an activation region, a deactivation region, and a potential region, based on the activated particles and the deactivated particles,
The active mass, the inactive mass, and the potential mass are calculated from the respective lattice communities based on the activation region, the inactive region, and the potential region of each lattice community,
Wherein the active state of each grid cluster is divided into the active state, the inactive state, and the latent state based on the calculated activation mass, inactivation mass, and potential mass of each grid community. An apparatus for mapping an environment around a vehicle. 6. The apparatus of claim 5,
Calculates a rotation angle and an operation distance of the unmanned vehicle during a predetermined time based on the moving speed and the rotation angle of the unmanned vehicle, calculates the position sources of the respective lattice communities based on the calculated rotation angle and the operation distance, And compensating for the position of each lattice community based on the location sources of the calculated lattice communities and calculating the dynamic state of each lattice community at the next time point based on the positions of the compensated lattice communities and predetermined discrete Gaussian noise And estimating an environment around the unmanned vehicle.

Images