KR102494926B1 - Apparatus and method for generating dynamic local map for operation of uav - Google Patents

Abstract

The present invention relates to a technique for generating a local dynamic map (LDM) for safe operation of an unmanned aerial vehicle (UAV). According to one aspect of the present invention, the unmanned aerial vehicle comprises: a sensor part scanning an environment around an unmanned aerial vehicle to generate scan data, and sensing the speed of the unmanned aerial vehicle; an LDM management part generating a local dynamic map expressing the environment around the unmanned aerial vehicle in a three-dimensional grid consisting of a plurality of voxels each representing an occupying probability and speed; and an operation control part controlling the operation of the unmanned aerial vehicle in accordance with the generated local dynamic map.

Description

APPARATUS AND METHOD FOR GENERATING DYNAMIC LOCAL MAP FOR OPERATION OF UAV

The present invention relates to a technique for generating a local dynamic map (LDM) for the safe operation of an unmanned aerial vehicle (UAV).

Accurate recognition of obstacles around the UAV is required for safe operation of the UAV. To this end, a technique for expressing the surrounding environment of the UAV with an Occupancy Grid map (OGM) has been conventionally proposed, but this has limitations in that it can only check occupancy and cannot recognize the shape of the obstacle or the speed of the obstacle. .

Korean Patent Publication No. 10-2015-0136209, "Multiple image-based obstacle avoidance system and method" Korean Patent Publication No. 10-2021-0065837, "Occupancy 　 Grid 　 Map 　 Generating Apparatus and Method"

It is an object of the present invention to provide techniques for generating local dynamic maps based on occupancy grids and particle filters.

Another object of the present invention is to provide a technique for generating a dynamic local map capable of recognizing the shape and movement of an obstacle.

It is also an object of the present invention to provide a technique for generating a dynamic local map that can reduce the usage of UAV hardware resources.

According to one aspect of the present invention, in a method for generating a local dynamic map that expresses a UAV surrounding environment as a three-dimensional grid composed of a plurality of voxels each representing an occupancy probability and a speed, a probability of occupancy greater than or equal to a reference value among a plurality of voxels is determined. predicting a position on a local dynamic map at time k, based on the velocity at time k-1 of at least one obstacle cluster composed of adjacent voxels having predicting an occupancy probability of each of the plurality of voxels at time k based on the predicted position of the at least one obstacle cluster; predicting the velocity at time k based on the velocity at time k-1 of the particles distributed on the local dynamic map; based on the predicted velocity of the particles, predicting the location on the local dynamic map of the particles at time k; generating scan data by scanning an environment around the UAV at time k; updating a predicted occupation probability of each of the plurality of voxels at time k based on the generated scan data; updating weights of particles at time k based on the updated occupancy probabilities; updating velocities of particles at time k using velocities of each of a plurality of voxels determined based on the predicted velocities of the particles and the updated weights; and regenerating at least one obstacle cluster composed of adjacent voxels having an occupancy probability greater than or equal to a reference value at time k. A method is provided.

In one embodiment, updating the position of the UAV at time k-1 based on the speed information of the UAV at time k-1; and regenerating the local dynamic map so that the coordinates at time k indicating the position of the UAV on the local dynamic map are the same as the coordinates at time k−1, wherein generating the scan data is such that the local dynamic map is Can be performed after regeneration.

In one embodiment, the size of the local dynamic map regenerated at time k may be the same as the size of the local dynamic map at time k-1.

In one embodiment, the positional difference between time k and time k−1 of the UAV may be an integer multiple of a voxel size.

In one embodiment, the number of particles at time k-1 is a preset number, and if the number of particles at time k is less than the preset number, performing resampling based on the weight of the particles to reach the preset number is further performed. can include

In addition, according to another aspect of the present invention, the sensor unit for generating scan data by scanning the environment around the UAV, and sensing the speed of the UAV; an LDM management unit that generates a local dynamic map expressing the surrounding environment of the UAV as a three-dimensional grid composed of a plurality of voxels each representing an occupancy probability and speed; and a navigation control unit controlling the operation of the UAV according to the generated local dynamic map, wherein the LDM management unit includes a time k-of at least one obstacle cluster composed of adjacent voxels having an occupancy probability equal to or greater than a reference value among a plurality of voxels. Based on the velocity at 1, predicting a position on the local dynamic map at time k, predicting an occupancy probability of each of a plurality of voxels at time k based on a predicted position of at least one obstacle cluster, and Based on the velocity at time k-1 of the particles distributed on the map, predicting the velocity at time k, predicting the location on the local dynamic map of the particles at time k based on the predicted velocity of the particles, At time k, the environment around the drone is scanned to generate scan data, and based on the generated scan data, a predicted occupancy probability of each of a plurality of voxels at time k is updated, and based on the updated occupancy probability, particles are generated at time k. update the weights in , and update the velocities of the particles at time k using the velocities of each of the plurality of voxels determined based on the predicted velocities of the particles and the updated weights, and have an occupancy probability equal to or greater than the reference value at time k An unmanned aerial vehicle regenerates at least one obstacle cluster composed of adjacent voxels.

According to an embodiment of the present invention, it is possible to create a local dynamic map based on the occupancy grid and particle filter.

In addition, according to another embodiment of the present invention, it is possible to create a dynamic local map capable of recognizing the shape and movement of an obstacle.

In addition, according to another embodiment of the present invention, it is possible to create a dynamic local map capable of reducing the usage of UAV hardware resources.

1 is a block diagram of an unmanned aerial vehicle according to an embodiment of the present invention;
2 is a flow diagram of a method for generating a local dynamic map according to an embodiment of the present invention;
3 is a diagram for explaining movement of an obstacle cluster according to an embodiment of the present invention;
4 is a diagram for explaining regeneration of a local dynamic map according to movement of an UAV according to an embodiment of the present invention;
5 is a block diagram of a method for generating a local dynamic map according to one embodiment of the present invention;
6 is a block diagram of a UAV according to another embodiment of the present invention;

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are only illustrated for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention These may be embodied in various forms and are not limited to the embodiments described herein.

Embodiments according to the concept of the present invention can apply various changes and can have various forms, so the embodiments are illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosures, and includes modifications, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component, for example, without departing from the scope of rights according to the concept of the present invention, a first component may be named a second component, Similarly, the second component may also be referred to as the first component.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle. Expressions describing the relationship between components, such as "between" and "directly between" or "directly adjacent to" should be interpreted similarly.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these examples. Like reference numerals in each figure indicate like elements.

1 is a block diagram of an unmanned aerial vehicle according to an embodiment of the present invention.

Referring to FIG. 1 , an unmanned aerial vehicle 1000 may include a sensor unit 1100, an LDM management unit 1200, and a driving control unit 1300.

The sensor unit 1100 may generate sensing data by sensing the surrounding environment of the UAV 1000 . In detail, the sensor unit 1100 may scan the surrounding environment of the UAV 1000 to generate scan data about the presence or absence of obstacles around the UAV 1000 .

In one embodiment, the sensor unit 1100 may be a LiDAR sensor, a radar sensor, or the like.

In addition, the sensor unit 1100 may generate movement data of the UAV 1000 by sensing the speed and direction of movement of the UAV 1000 .

In one embodiment, the sensor unit 1100 may be a gyro sensor or an acceleration sensor.

The LMD management unit may create a local dynamic map of the surrounding environment of the UAV 1000 capable of recognizing the presence of an obstacle based on the occupancy grid and particle filter.

The local dynamic map may be expressed as a 3D grid composed of a plurality of voxels. Each voxel expresses the occupancy probability and speed in the form of a cube and may correspond to specific coordinates on a local dynamic map. Here, the velocity may be a vector including position, velocity, and direction on the local dynamic map.

In one embodiment, the local dynamic map may represent an obstacle cluster for which clustering of two or more voxels has been performed. The obstacle cluster may be recognized as an obstacle existing in the surrounding environment of the UAV 1000 . The obstacle cluster may be a fixed position obstacle or an obstacle moving at a specific speed.

In one embodiment, an obstacle cluster may be composed of two or more voxels having an occupancy probability equal to or greater than a preset reference value, and voxels constituting the obstacle cluster may be composed of voxels at a preset distance from each other, constituting a cluster. The number of voxels used may be a preset number. That is, the obstacle cluster may be composed of a preset number of voxels adjacent to each other having a refining probability equal to or greater than a reference value.

Hereinafter, a detailed description of LDM generation will be given with reference to FIGS. 2 to 5 .

The driving controller 1300 may control the driving of the UAV 1000 such as avoiding an obstacle based on the generated local dynamic map.

2 is a flowchart of a method for generating a local dynamic map according to an embodiment of the present invention.

Hereinafter, the method shown in FIG. 2 will be described below with an example of being performed by the UAV 1000 shown in FIG.

In step S2100, the position of the obstacle cluster is predicted. Specifically, the UAV 1000 may predict the position of the obstacle cluster at time k based on the LDM at time k-1. Here, the position of the obstacle cluster at time k can be predicted based on the position and speed of the obstacle cluster included in the LDM at time k-1. In this case, the speed of the obstacle cluster may be determined as an average speed of voxels constituting the obstacle cluster.

In one embodiment, referring to FIG. 3 , an example of movement of an obstacle cluster on a local dynamic map with the location of the UAV 1000 as the center coordinate is shown. For clarity of explanation, a case where the local dynamic map is a 2D map is described as an example, but it is obvious that the same applies to a 3D map as well. At time k-1, obstacle cluster 1 consisting of 5 voxels and having speed in the upper right direction is shown in the upper left corner of the local dynamic map at time k-1, and obstacle cluster 2 consisting of 3 voxels and having no speed is shown in the lower right corner. is shown Obstacle cluster 1 has a velocity toward the upper right at t-1, so based on the time interval between t-1 and t and the velocity of the obstacle cluster at t-1, we predict that the position of the obstacle cluster will move to the upper right at t it becomes possible to do In addition, since the obstacle cluster 2 does not have a velocity at time k-1, it is possible to predict that it will maintain the same position at time k. Here, the speed of each obstacle cluster may mean the average speed of voxels constituting each obstacle cluster.

In step S2200, the occupation probability of the LDM at time k is predicted. Specifically, the UAV 1000 may predict the occupancy probability at time k based on the predicted position of the obstacle cluster.

In one embodiment, as shown in FIG. 3, since obstacle cluster 1 moves to the upper right corner and obstacle cluster 2 does not move, coordinates corresponding to voxels constituting obstacle cluster 1 are occupied at time k-1. By reflecting the probability as the occupation probability of coordinates corresponding to the voxels constituting the obstacle cluster 1 at time k, the occupation probability of each of a plurality of voxels constituting the LDM at time k can be predicted.

In step S2300, particles are predicted. Specifically, the UAV 1000 may predict the velocity and location of particles at time k-1 based on the LDM at time k-1. Here, since the particles are distributed in the LDM, a plurality of particles may exist in one voxel, and the speed and position of each particle may be different.

In one embodiment, the prediction of particles is performed in the paper “Hybrid sampling bayesian occupancy filter. 2014 IEEE Intelligent Vehicles Symposium Proceedings. IEEE, 2014” can be used.

In one embodiment, the velocity of a particle can be predicted according to Equation 1.

는 시간 k-1에서 속도

에

(zero mean normal distribution noise with covariance Σ를 더한 값으로 예측될 수 있다. That is, the velocity at time k of the ith particle

is the velocity at time k-1

to

(It can be predicted by adding zero mean normal distribution noise with covariance Σ.

In one embodiment, the position of the particle may be predicted according to Equation 2.

는 시간 k-1에서 위치

에다가 파티클의 예측된 시간 k에서 속도와 시간 k-1과 시간 k간의 차이를 곱한 값을 더함으로써 예측될 수 있다. That is, the position at time k of the I-th particle is

is the position at time k-1

It can be predicted by adding the product of the velocity at the predicted time k of the particle and the difference between time k - 1 and time k.

In step S2400, the location of the UAV 1000 is updated. Specifically, the UAV 1000 may update the location and speed on the local dynamic map at time k based on the location and speed on the local dynamic map of the UAV 1000 at time k−1. However, at this time, the UAV 1000 may regenerate the local dynamic map so that the position of the UAV 1000 at time k and at time k−1 has the same coordinates on the local dynamic map.

For example, referring to FIG. 4 , when the UAV 1000 is located at the coordinates of voxel 0, which is the center coordinate, on the local dynamic map at time k-1, but moves to voxel 6 at time k. For example, the UAV 1000 may regenerate the local dynamic map so that voxel 6 becomes the center coordinate of the local dynamic map again. At this time, the information of voxels 13, 8, 3, 23, 18, 19, 15, 16, and 17 included in the local dynamic map at k-1 disappears and information about the area around the newly added drone 1000 at time k disappears. It can be reconstructed into voxels. At this time, the UAV 1000 may use a circular buffer array to reduce memory usage and speed data processing.

Also, the movement of the UAV 1000 may be determined as an integer multiple of a unit of voxels constituting the local dynamic map, without considering the rotation of the UAV 1000 .

In step S2500, the occupation probability of the LDM is updated. Specifically, the UAV 1000 may scan the surrounding environment at time k to generate scan data, and update the occupancy probability for each of a plurality of voxels constituting the LDM based on the generated scan data. .

In step S2600, particles are updated. Specifically, the UAV 1000 may update particles based on the updated occupancy probability at time k.

In an embodiment, the weights of the particles at time k may be updated by adding the updated occupancy probability of the voxel corresponding to the location where each particle is distributed to the weights of the particles at time k−1.

In an embodiment, the velocity of each particle may be updated according to Equation 3 based on the velocity of the voxel corresponding to the location where each particle is distributed.

는 시간 k에서 j 번째 복셀의 속도

에

(zero mean normal distribution noise with covariance Σ을 더함으로써 산출될 수 있다. i.e. the velocity of the updated ith particle

is the velocity of the jth voxel at time k

to

(It can be calculated by adding zero mean normal distribution noise with covariance Σ.

는 수학식 4에 따라 산출될 수 있다. where, the velocity of the jth voxel at time k

Can be calculated according to Equation 4.

는 각 복셀에 대응하는 각 파티클들의 시간 k에서 예측된 속도와 업데이트된 가중치의 곱들의 합을, 각 복셀에 대응하는 각 파티클들의 가중치들의 합으로 나눔으로써 산출될 수 있다.That is, the velocity of the jth voxel at time k

may be calculated by dividing the sum of products of the predicted velocity and the updated weight at time k of each particle corresponding to each voxel by the sum of the weights of each particle corresponding to each voxel.

In step S2700, an obstacle cluster is regenerated. Specifically, the UAV 1000 may generate an obstacle cluster by clustering adjacent voxels having an occupation probability equal to or greater than a reference value based on the occupation probability of each voxel at time k.

In step S2700, the particles are resampled. Specifically, in order to maintain the number of particles at a preset number, the UAV 1000 performs resampling based on the updated weight at time k when the number of particles decreases according to the movement of the UAV 1000 or the movement of an obstacle, The number of particles may be maintained at a preset number.

Also, in the initial stage of generating the dynamic local map, the occupancy probability of each voxel is set to 0.5, the particles are evenly distributed on the dynamic local map, the velocities are randomly distributed, and the weights are equally set.

5 is a block diagram for explaining a method for generating a local map according to an embodiment of the present invention.

Referring to FIG. 5, the local dynamic map at time k-1 generates an obstacle cluster by cluttering two or more voxels based on the occupancy probability at k-1, and the velocity of the cluster is based on the velocity of resampled particles. It is decided. According to the movement of the obstacle cluster at time k, occupancy probabilities of a plurality of voxels constituting the local dynamic map are predicted, and the positions and velocities of the particles are predicted. Then, when the UAV 1000 moves, a local dynamic map is regenerated according to the movement of the UAV 1000. The UAV 1000 scans the surrounding environment at time k and updates occupancy probabilities of a plurality of voxels constituting a local dynamic map based on the scan data. Based on the updated occupancy probability, the velocity and weight of the particles may be updated. Particles may be resampled based on the updated weight to maintain the number of particles. An obstacle cluster may be generated by clustering two or more voxels based on the occupancy probability of the voxels at time k. Based on the position and speed of the cluster at time k and the position, speed, and weight of the particles, the position and speed of the obstacle cluster may be determined at time k+1 by iterating again.

6 is a block diagram of a UAV according to another embodiment of the present invention.

As shown in FIG. 6, the UAV 1000 includes at least one element of a processor 6100, a memory 6200, a storage unit 6300, a user interface input unit 6400, and a user interface output unit 6500. and they can communicate with each other through the bus 6600. In addition, the UAV 1000 may also include a network interface 6700 for accessing a network. Processor 6100 may be a CPU or semiconductor device that executes processing instructions stored in memory 6200 and/or storage 6300 . The memory 6200 and the storage unit 6300 may include various types of volatile/nonvolatile storage media. For example, the memory may include ROM 6240 and RAM 6250.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

1000: drone 1100: sensor unit
1200: LDM management unit 1300: operation control unit

Claims (10)

A method for generating a local dynamic map expressing a UAV surrounding environment as a three-dimensional grid composed of a plurality of voxels each representing an occupancy probability and speed, the method comprising:
predicting a location on the local dynamic map at time k, based on a velocity at time k-1 of at least one obstacle cluster composed of adjacent voxels having an occupancy probability equal to or greater than a reference value among the plurality of voxels;
predicting an occupation probability of each of the plurality of voxels at the time k based on the predicted position of the at least one obstacle cluster;
predicting velocities at time k based on the velocities of particles distributed on the local dynamic map at time k-1;
based on the predicted velocity of the particles, predicting the location on the local dynamic map of the particles at the time k;
generating scan data by scanning an environment around the UAV at the time k;
updating a predicted occupation probability of each of the plurality of voxels at the time k based on generated scan data;
updating weights of the particles at the time k based on the updated occupancy probabilities;
updating the velocities of the particles at the time k using the velocities of each of the plurality of voxels determined based on the predicted velocities of the particles and the updated weights; and
regenerating at least one obstacle cluster composed of adjacent voxels having an occupancy probability greater than or equal to the reference value at the time k;
Including. method.
According to claim 1,
updating the location of the UAV at the time k-1 based on the speed information of the UAV at the time k-1; and
Regenerating the local dynamic map so that the coordinates at the time k representing the location of the UAV on the local dynamic map are the same as the coordinates at the time k-1,
Characterized in that the generating of the scan data is performed after the local dynamic map is regenerated.
According to claim 2,
characterized in that the size of the local dynamic map regenerated at the time k is the same as the size of the local dynamic map at the time k-1.
According to claim 2,
Characterized in that the position difference between the time k and the time k-1 of the UAV is an integer multiple of a voxel size.
According to claim 2,
The step of performing resampling based on the weight of the particles so that the number of particles at the time k−1 is a preset number and the number of particles at the time k is less than the preset number reaches the preset number; Including, how.
a sensor unit that scans the environment around the UAV to generate scan data and senses the speed of the UAV;
an LDM management unit that generates a local dynamic map expressing the surrounding environment of the UAV as a three-dimensional grid composed of a plurality of voxels each representing an occupancy probability and speed; and
Including a driving control unit that controls the operation of the UAV according to the generated local dynamic map,
The LDM management unit,
Based on the velocity at time k-1 of at least one obstacle cluster composed of adjacent voxels having an occupancy probability equal to or greater than a reference value among the plurality of voxels, predicting a position on the local dynamic map at time k;
predicting an occupation probability of each of the plurality of voxels at the time k based on the predicted position of the at least one obstacle cluster;
Based on the velocities of the particles distributed on the local dynamic map at the time k-1, predicting the velocities at the time k;
Based on the predicted velocity of the particles, predict the location on the local dynamic map of the particles at the time k;
At the time k, the environment around the UAV is scanned to generate scan data;
Updating a predicted occupation probability of each of the plurality of voxels at the time k based on the generated scan data;
Update weights of the particles at the time k based on the updated occupancy probability;
The velocity of the particles is updated at the time k using the velocity of each of the plurality of voxels determined based on the predicted velocity of the particles and the updated weight,
At the time k, the UAV regenerates at least one obstacle cluster composed of adjacent voxels having an occupancy probability greater than or equal to the reference value.
According to claim 6,
The LDM management unit,
At the time k-1, the location of the UAV is updated based on the speed information of the UAV at the time k-1, and the coordinates at the time k representing the location of the UAV on the local dynamic map are the coordinates at the time k-1. Regenerate the local dynamic map to be the same as the coordinates of 1,
Characterized in that the scan data is generated after the local dynamic map is regenerated.
According to claim 7,
Characterized in that the size of the local dynamic map regenerated at the time k is the same as the size of the local dynamic map at the time k-1.
According to claim 6,
Characterized in that the position difference between the time k and the time k-1 of the UAV is an integer multiple of a voxel size.
According to claim 6,
The LDM management unit,
The number of particles at the time k-1 is a preset number, and if the number of particles at the time k is less than the preset number, resampling is performed based on the weight of the particles to reach the preset number. UAV.

Images