KR102259920B1 - Estimation of azimuth angle of unmanned aerial vehicle that operates in indoor environment - Google Patents

Abstract

A method for estimating an azimuth of an unmanned aerial vehicle (UAV) operating in an indoor environment is provided. The disclosed method comprises: a step of calculating an azimuth based on magnetic field measurement data received from a geomagnetic sensor of a UAV; a step of determining a sensor-based azimuth from the calculated azimuth and a preset error compensation value; a step of detecting a predetermined characteristic of an indoor environment based on image data received from an imaging unit of the UAV; a step of determining an image-based azimuth from the detected predetermined characteristic; and a step of updating the error compensation value based on the sensor-based azimuth and the image-based azimuth.

Description

ESTIMATION OF AZIMUTH ANGLE OF UNMANNED AERIAL VEHICLE THAT OPERATES IN INDOOR ENVIRONMENT

The present disclosure relates to estimation of an azimuth of an unmanned aerial vehicle (UAV) operating in an indoor environment.

Research on the use of unmanned vehicles, particularly unmanned aerial vehicles (UAVs) such as drones, is active in various environments. In fact, in some environments in which the UAV will operate, it may be difficult to accurately estimate the posture of the UAV. For example, when an unmanned mobile vehicle is operated in an indoor environment where it is not easy to use a satellite navigation system such as a global positioning system (GPS), the navigation system of the unmanned mobile object is a geomagnetic sensor (which The azimuth of the unmanned moving object may be estimated using magnetic field measurement data provided by a magnetometer (also referred to as a magnetometer). However, compared to other sensors commonly used in various inertial navigation systems, for example, a gyroscope or an accelerometer, the geomagnetic sensor is greatly affected by the surrounding environment. For example, distortion may occur in magnetic field measurement data output from the geomagnetic sensor due to a surrounding object that causes a magnetic force other than the Earth's magnetic force or a surrounding object that changes the direction of an existing magnetic force.

Estimation of the azimuth of a UAV operating in an indoor environment is disclosed herein.

In Example 1, a method for estimating an azimuth of an unmanned aerial vehicle (UAV) operating in an indoor environment includes: the azimuth angle based on magnetic field measurement data received from a geomagnetic sensor of the UAV calculating ; determining a sensor-based azimuth from the calculated azimuth and a preset error compensation value; detecting a predetermined characteristic of the indoor environment based on image data received from an imaging unit of the UAV; determining an image-based azimuth from the detected predetermined feature; and updating the error compensation value based on the sensor-based azimuth and the image-based azimuth.

Example 2 includes the subject matter of Example 1, wherein the predetermined characteristic includes a straight-line portion having a predetermined absolute azimuth, and wherein determining the image-based azimuth includes a relative of the straight-line portion to the UAV. and calculating the image-based azimuth from the angle and the absolute azimuth.

Example 3 includes the subject matter of Example 1 or Example 2, wherein updating the error compensation value includes: filtering the sensor-based azimuth and the image-based azimuth with a complementary filter to calculate a filtered azimuth; , setting the error compensation value as a value in which the error compensation value is accumulated in a difference between the sensor-based azimuth and the filtered azimuth.

Example 4 includes the subject matter of any of Examples 1-3, wherein updating the error compensation value is based on whether a difference between the sensor-based azimuth and the image-based azimuth is greater than or equal to a threshold value; and setting as an initial value or a value in which the error compensation value is accumulated in the difference.

Example 5 includes the subject matter of any of Examples 1-4, further comprising setting the error compensation value to an initial value when the image data is not received from the imaging unit for more than a threshold time.

Example 6 includes the subject matter of any of Examples 1-5, wherein the method further comprises performing a calibration of the geomagnetic sensor based on data collected from a two-dimensional in-plane rotation of the geomagnetic sensor. and calculating the azimuth includes calculating the azimuth based on the magnetic field measurement data to which the calibration is applied.

Example 7 includes the subject matter of any of Examples 1-6, wherein calculating the azimuth comprises calculating the azimuth as a gyroscope based azimuth based on angular velocity measurement data received from a gyroscope of the UAV. calculating the azimuth as a geomagnetic sensor-based azimuth based on the magnetic field measurement data, and calculating the azimuth by filtering the gyroscope-based azimuth and the geomagnetic sensor-based azimuth with a complementary filter. .

In Example 8, there is provided a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a computer processor, cause the computer processor to perform the method described in any of Examples 1-7.

In Example 9, a computing device includes a processor and a memory, wherein the memory comprises a set of computer program instructions executable by the processor to estimate an azimuth of an unmanned aerial vehicle (UAV) operating in an indoor environment. encoded, the set comprising: instructions for calculating an azimuth of the UAV based on magnetic field measurement data received from a geomagnetic sensor of the UAV; instructions for determining a sensor-based azimuth from the calculated azimuth and a preset error compensation value; instructions for detecting a predetermined characteristic of the indoor environment based on image data received from the imaging unit of the UAV; instructions for determining an image-based azimuth from the detected predetermined feature; and a command for updating the error compensation value based on the sensor-based azimuth and the image-based azimuth.

Example 10 includes the subject matter of Example 9, wherein the predetermined characteristic comprises a straight-line portion having a predetermined absolute azimuth, and wherein the instructions to determine the image-based azimuth include: a relative angle of the straight-line portion to the UAV and the absolute and instructions for calculating the image-based azimuth from the azimuth.

Example 11 includes the subject matter of Examples 9 or 10, wherein the command for updating the error compensation value includes: a command for filtering the sensor-based azimuth and the image-based azimuth with a complementary filter to calculate a filtered azimuth; and a command for setting a value as a value in which the error compensation value is accumulated in a difference between the sensor-based azimuth and the filtered azimuth.

Example 12 includes the subject matter of any of Examples 9-11, wherein the instruction to update the error compensation value is based on whether a difference between the sensor-based azimuth and the image-based azimuth is greater than or equal to a threshold value: and a command for setting α as an initial value or a value in which the error compensation value is accumulated in the difference.

Example 13 includes the subject matter of any of Examples 9-12, wherein the set further includes instructions to set the error compensation value to an initial value when the image data is not received from the imaging unit for more than a threshold time. include

Example 14 includes the subject matter of any of Examples 9-13, wherein the set further comprises instructions to perform a calibration of the geomagnetic sensor based on data collected from a two-dimensional in-plane rotation of the geomagnetic sensor; , The command for calculating the azimuth includes a command for calculating the azimuth based on the magnetic field measurement data to which the calibration is applied.

Example 15 includes the subject matter of any of Examples 9-14, wherein the instructions for calculating the azimuth include: calculating the azimuth as a gyroscope based azimuth based on angular velocity measurement data received from a gyroscope of the UAV; , calculating the azimuth as a geomagnetic sensor-based azimuth based on the magnetic field measurement data, and filtering the gyroscope-based azimuth and the geomagnetic sensor-based azimuth with a complementary filter to calculate the azimuth.

In Example 16, an unmanned aerial vehicle comprising the geomagnetic sensor, imaging unit, and computing device described in any of Examples 9-15 is provided.

The previous summary is provided to introduce in a simplified form some aspects that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to delineate the scope of the claimed subject matter. Furthermore, claimed subject matter is not limited to implementations that provide any or all advantages discussed herein.

According to the present disclosure, in estimating the azimuth of a UAV operating in an indoor environment, it is possible to improve the accuracy of estimation by utilizing not only the magnetic field measured by the geomagnetic sensor but also an image in which predetermined features in the indoor environment are captured.

1 is a block diagram showing an example of a navigation system of an exemplary Unmanned Aerial Vehicle (UAV) for operation in an indoor environment.
FIG. 2 shows an exemplary flow of posture estimation of the UAV of FIG. 1 .
FIG. 3 shows an exemplary flow of image processing for calculating the azimuth of the UAV of FIG. 1 .
4A and 4B are diagrams for explaining edge detection and straight line detection in the image processing of FIG. 3 , respectively.
FIG. 5 is a diagram for explaining an azimuth of the UAV of FIG. 1 .
FIG. 6 is a diagram for comparing the output data of the geomagnetic sensor used to calculate the azimuth of the UAV of FIG. 1 before and after calibration of the geomagnetic sensor is applied.
7 shows an exemplary flow of compensation for an error in azimuth estimation of the UAV of FIG. 1 .
8 shows another exemplary flow of compensation for an error in azimuth estimation of the UAV of FIG. 1 .
9 is a flowchart illustrating an example of a process for estimating an azimuth of the UAV of FIG. 1 .

Various terms used in the present disclosure are selected from the terminology of common terms in consideration of their functions in this document, which may be differently recognized according to the intention of those skilled in the art, practice, or emergence of new technology. In certain instances, some terms may be given meanings as set forth in the Detailed Description. Accordingly, terms used in this document should be defined consistent with their meaning in the context of the present disclosure and not simply by their names.

In this document, the terms "comprise", "have" and the like are used when specifying the presence of an element listed hereinafter, such as a certain feature, number, step, operation, element, information, or combination thereof. Unless otherwise indicated, these terms and variations thereof are not intended to exclude the presence or addition of other elements.

As used herein, the terms “first,” “second,” and the like are intended to identify several elements that resemble each other. Unless otherwise stated, such terms are not intended to impose limitations, such as the specific order of these elements or their use, but are merely used to refer to the various elements separately. For example, an element may be referenced in one example by the term "first" while the same element may be referenced in another example by a different ordinal number, such as "second" or "third." In such instances, these terms do not limit the scope of the present disclosure. Also, use of the term "and/or" in a list of multiple elements includes all possible combinations of those listed, including any one or multiple of those listed. Furthermore, expressions in the singular form include the meaning of the plural unless clearly used otherwise.

Certain examples of the present disclosure will now be described in detail with reference to the accompanying drawings. However, the present disclosure may be embodied in many different forms and should not be construed as being limited to the examples presented in this document. Rather, these examples are given to provide a better understanding of the scope of the present disclosure.

1 is a block diagram illustrating an example of a navigation system 100 of an exemplary UAV 10 for operation in an indoor environment. Examples of UAV 10 include drones with one or more rotors (or rotor blades), drones with fixed wings, and drones with a tilt-rotor mechanism. do.

The exemplary navigation system 100 of FIG. 1 provides current navigation information (which indicates, for example, the position, speed, and attitude of the UAV 10 ) to measurement information from sensors mounted on the UAV 10 and previous navigation information. Calculate based on information. In some examples, the navigation system 100 may not have a mechanism to receive and use assistance information from an external navigation system (eg, GPS).

In the example of FIG. 1 , the navigation system 100 includes a gyroscope 110 , an accelerometer 120 , a geomagnetic sensor 130 , an imaging unit 140 , and a processing unit 150 . do. Other example implementations of the navigation system 100 are also contemplated. For example, the navigation system 100 may also include additional components not shown and/or may include some but not all of the components shown in FIG. 1 .

,

및

를 나타내도록 UAV에 장착될 수 있다. 예를 들어, 자이로스코프(140)는 3축 자이로스코프일 수 있다.In the example shown, gyroscope 110 provides angular velocity measurement data. In an exemplary implementation, the gyroscope 110 is configured so that the angular velocity measurement data is the measured angular velocity with respect to the x-axis, y-axis, and z-axis of the body coordinate system of the UAV 10, respectively.

,

and

It can be mounted on a UAV to indicate For example, the gyroscope 140 may be a three-axis gyroscope.

,

및

를 나타내도록 UAV(10)에 장착될 수 있다. 예를 들어, 가속도계(150)는 3축 가속도계일 수 있다.In the illustrated example, accelerometer 120 provides acceleration measurement data. In an exemplary implementation, the accelerometer 120 provides the acceleration measurement data for measured accelerations with respect to the x-axis, y-axis, and z-axis, respectively.

,

and

It may be mounted on the UAV 10 to indicate For example, the accelerometer 150 may be a three-axis accelerometer.

,

및

을 나타내도록 UAV(10)에 장착될 수 있다. 예를 들어, 지자기 센서(110)는 3축 자력계일 수 있다.In the illustrated example, the geomagnetic sensor 130 provides magnetic field measurement data. In an exemplary implementation, the geomagnetic sensor 110 is a magnetic field measurement data measured in the x-axis, y-axis and z-axis directions, respectively.

,

and

It may be mounted on the UAV 10 to indicate For example, the geomagnetic sensor 110 may be a three-axis magnetometer.

In the illustrated example, the imaging unit 140 provides image data. In an example implementation, the imaging unit 120 may be mounted to the UAV 10 to provide image data of M×N pixels captured along the forward direction (eg, the x-axis direction) of the UAV 10 at a predetermined period. have. For example, the imaging unit 120 may be a CMOS image sensor, a camera module, or other type of imaging unit.

In the illustrated example, the processing unit 150 estimates the pose of the UAV 10 . An exemplary flow of such pose estimation is shown in FIG. 2 .

,

및

으로서 계산할 수 있다.In some example implementations, as represented by blocks 210 and 215 , processing unit 150 determines the roll angle of UAV 10 based on angular velocity measurement data received from gyroscope 110 , The pitch angle and the yaw angle (i.e., azimuth) are respectively calculated according to the following equations:

,

and

can be calculated as

As can be seen from the above equation, calculating the posture of the UAV 10 as the angular velocity measurement data of the gyroscope 110 involves integration causing accumulation of errors. The divergence of the calculated value due to such drift can be suppressed, for example, as described below.

및

으로서 계산할 수 있다.In some example implementations, as represented by block 220 , the processing unit 150 calculates the roll angle and pitch angle of the UAV 10 based on the accelerometer measurement data received from the accelerometer 120 by the following equations: according to each

and

can be calculated as

및 피치 각

을 가속도 측정 데이터를 기반으로 계산된 롤 각

및 피치 각

과 각각 융합하여 UAV(10)의 센서 기반 롤 각

및 센서 기반 피치 각

을 산출할 수 있다. 특정한 예에서, 그러한 융합은 상보 필터(complementary filter)의 사용, 예컨대, 자이로스코프 기반 각도 값

및

에 고대역 통과 필터(High Pass Filter: HPF)를 적용하는 것 및 가속도계 기반 각도 값

및

에 저대역 통과 필터(Low Pass Filter: LPF)를 적용하는 것을 포함할 수 있다.Then, as represented by block 225 , processing unit 150 performs the calculated roll angle based on the angular velocity measurement data.

and pitch angle

Roll angle calculated based on the accelerometer data

and pitch angle

The sensor-based roll angle of the UAV (10) by fusion with

and sensor-based pitch angle

can be calculated. In certain instances, such fusion may be accomplished using a complementary filter, eg, a gyroscope based angle value.

and

Applying a High Pass Filter (HPF) to the accelerometer-based angle values

and

It may include applying a low pass filter (LPF) to the.

및 센서 기반 피치 각

에 또한 기반하여, UAV(10)의 방위각을 다음 수학식에 따라

으로서 계산할 수 있다.Further, in some example implementations, as represented by block 230 , the processing unit 150 is configured to perform the roll angle and pitch of the UAV 10 and based on the magnetic field measurement data received from the geomagnetic sensor 130 . angle, e.g. sensor-based roll angle

and sensor-based pitch angle

Also based on , the azimuth of the UAV 10 is calculated according to the following equation

can be calculated as

을 자기장 측정 데이터를 기반으로 계산된 방위각

과 융합할 수 있다. 특정한 예에서, 그러한 융합은 상보 필터의 사용, 예컨대, 자이로스코프 기반 각도 값

에 HPF를 적용하는 것 및 지자기 센서 기반 각도 값

에 LPF를 적용하는 것을 포함할 수 있다.Then, as represented by block 235 , processing unit 150 performs the calculated azimuth based on the angular velocity measurement data.

azimuth calculated based on magnetic field measurement data

can be fused with In certain instances, such fusion may be achieved by the use of a complementary filter, eg, a gyroscope based angle value.

Applying HPF to and geomagnetic sensor based angle values

may include applying LPF to

및 방위각

의 융합은 사전설정된 오차 보상 값

으로써 오차 보상이 될 수 있다. 예를 들어, 블록(250)에 의해 나타내어진 바와 같이, 처리 유닛(150)은 융합으로부터 초래된 값에서 오차 보상 값

을 감산함으로써 UAV(10)의 방위각의 최종적인 추정치로서 센서 기반 방위각

을 산출할 수 있다. 특정한 예에서, 처리 유닛(150)은 UAV의 초기화 동안에 오차 보상 값

을 초기 값(가령, 0)으로 초기화할 수 있고, 블록(245)에 의해 나타내어진 바와 같이, 오차 보상 값

을 UAV(10)의 센서 기반 방위각

및 영상 기반 방위각

에 기반하여 갱신할 수 있다. 영상 기반 방위각

은, 블록(240)에 의해 나타내어진 바와 같이, 촬상 유닛(140)으로부터 수신된 영상 데이터에 대해 처리 유닛(150)에 의해 수행된 영상 처리의 출력일 수 있다.Further, in some example implementations, azimuth

and azimuth

fusion of the preset error compensation value

This can be error compensation. For example, as represented by block 250 , processing unit 150 may set an error compensation value in a value resulting from the fusion.

The sensor-based azimuth as a final estimate of the azimuth of the UAV 10 by subtracting

can be calculated. In a particular example, the processing unit 150 sets the error compensation value during initialization of the UAV.

may be initialized to an initial value (eg, 0), and an error compensation value, as represented by block 245 .

to the sensor-based azimuth of the UAV (10).

and image-based azimuth

can be updated based on image based azimuth

may be an output of image processing performed by the processing unit 150 on the image data received from the imaging unit 140 , as indicated by block 240 .

이 새로 출력될 때까지, 위와 같이 계산된 오차 보상 값

이 센서 기반 방위각

의 산출에 사용될 수 있다.In some examples, the processing unit 150 may receive the magnetic field measurement data from the geomagnetic sensor 130 in a first period, and may receive image data from the imaging unit 140 in a second period, in a second period (eg, , 0.5 sec) may be greater than the first period (eg, 0.01 sec). In these examples, until new image data is received, eventually the image based azimuth

Until the new output, the error compensation value calculated as above

This sensor based azimuth

can be used to calculate

및 지자기 센서 기반 각도 값

의 융합에 오차 보상을 적용하는 것은 보다 신뢰성 있는 방위각 추정을 가능하게 한다.The above-described error compensation may suppress the influence of distortion that may occur in the magnetic field measurement data on the azimuth estimation. As such, even when experiencing significant geomagnetic disturbance while driving the UAV 10 indoors, the gyroscope-based angle value

and geomagnetic sensor-based angle values

Applying error compensation to the fusion of α allows more reliable azimuth estimation.

The following provides additional details regarding the pose estimation illustrated in FIG. 2 .

First, referring to FIG. 3 , an exemplary flow of image processing for calculating the azimuth of the UAV 10 is discussed. Blocks 310 to 360 of FIG. 3 may be included in block 240 of FIG. 2 .

In some example implementations, as represented by block 310 , the processing unit 150 is configured to convert image data captured by the imaging unit 140 while the UAV 10 is traveling in an indoor environment to the imaging unit 140 . ) can be received from Within the indoor environment, for example, on the floor of the indoor environment, there may be a predetermined feature comprising a marked straight portion (e.g., a straight artefact), whether visually or otherwise recognizable (e.g., detectable with an infrared camera). have.

In some example implementations, the received image data may be image data in RGB color format. Then, as represented by block 320 , the processing unit 150 may convert the RGB colors of the received image data to grayscale.

In some example implementations, as represented by block 330 , processing unit 150 may process the grayscale image data with an edge-preserving filter. For example, the edge preservation filter may be a bilateral filter suitable for increasing image sharpness and reducing image noise.

In some example implementations, as represented by block 340 , processing unit 150 may detect an edge from the filtered image data. For example, a Canny edge detection algorithm may be used for such edge detection. 4A shows the result of edge detection.

In some example implementations, as represented by block 350 , processing unit 150 may detect a marked straight-line portion within the indoor environment based on the detected edge. For example, a Hough line detection algorithm may be used for such straight line detection. Fig. 4B shows the result of linear detection.

을 산출할 수 있다. 예를 들어, 도 5를 참조하면, 직선 부분이 검출된 경우에, 처리 유닛(150)은 UAV(10)의 전진 방향, 예컨대, 영상 데이터의 수직 해상도가 정의되는 수직 영상 방향(가령, 도 4b의 영상의 위쪽 방향)에 대한 직선 부분의 상대 각도

를 판정할 수 있다. 한편, 직선 부분의 자북 기준 절대 방위각

은 사전결정된 값으로서 처리 유닛(150)에 주어질 수 있다. 그러면, 처리 유닛(150)은 직선 부분의 절대 방위각

및 상대 각도

를 기반으로 영상 기반 방위각

을 산출할 수 있다. 예를 들어, 도 5의 예에서, UAV(10)의 영상 기반 방위각은

로 주어진다.In some example implementations, as represented by block 360 , processing unit 150 is configured to perform image-based azimuth of UAV 10 based on the detected straight-line portion.

can be calculated. For example, referring to FIG. 5 , when a straight line portion is detected, the processing unit 150 determines the forward direction of the UAV 10 , eg, a vertical image direction in which the vertical resolution of image data is defined (eg, FIG. 4B ). the relative angle of the straight part with respect to the upward direction of the image of

can be determined. On the other hand, the absolute azimuth relative to magnetic north of the straight line

may be given to the processing unit 150 as a predetermined value. Then, the processing unit 150 determines the absolute azimuth angle of the straight line portion.

and relative angle

based on image based azimuth

can be calculated. For example, in the example of FIG. 5 , the image-based azimuth of the UAV 10 is

Is given by

Next, an example of calibration of the geomagnetic sensor 130 is discussed. The magnetic field measurement data provided by the geomagnetic sensor 130 may be distorted due to surrounding objects in the operating environment in addition to the measurement value of the Earth's magnetic field. Correcting the distortion in the magnetic field measurement data is to remove the influence of magnetic force, not the Earth's magnetic force, from the magnetic field measurement data, and is particularly useful for estimating the initial azimuth of the UAV 10 . Calibration of geomagnetic sensors is the process of determining the parameters to be used for such calibration.

In some example implementations, the processing unit 150 may collect magnetic field measurement data from a two-dimensional planar rotation of the geomagnetic sensor 130 . In particular, considering that the UAV 10 can keep the roll angle and pitch angle as 0 degrees as possible during hovering, such a two-dimensional calibration helps in azimuth estimation without significant degradation compared to a more complex three-dimensional calibration. can be

For example, processing unit 150 may control UAV 10 (the geomagnetic sensor 130 of) to rotate 360 degrees on the x-y plane during initialization. The data collected from such rotation may include a set of horizontal magnetic vectors representing the magnetic forces in the x- and y-axis directions. Then, the processing unit 150 may determine the parameter based on the collected data and use the parameter to correct distortion in the future magnetic field measurement data. 6 shows before and after such calibration is applied to the magnetic field measurement data output from the geomagnetic sensor 130 . The data collected from the two-dimensional plane rotation of the geomagnetic sensor 130 is depicted as a graph closer to a circle centered on the origin, as the correction of the distortion is more accurate. If the correction of the distortion is incorrect, the center of the pie graph will remain off the origin due to hard iron distortion and/or the pie graph will be distorted and remain elliptical due to soft iron distortion.

In some example implementations, the two-dimensional calibration described above uses an elliptic equation matrix representation of the collected magnetic field measurement data. The coefficients of such an elliptic equation depend on the error parameter of the geomagnetic sensor 130 . Thus, such a calibration involves performing an eigen-decomposition (also referred to as diagonal decomposition) on the represented matrix, and calculating the coefficients of the elliptic equation by means of a linear least squares method, and eventually the geomagnetic sensor. It involves determining an error parameter of (130).

Further, with reference to FIG. 7 , an exemplary flow of compensation for errors in azimuth estimation of the UAV 10 is discussed. Blocks 710 to 730 of FIG. 7 may be included in block 245 of FIG. 2 .

및 영상 기반 방위각

을 융합하여 필터링된 방위각

을 산출할 수 있다. 특정한 예에서, 그러한 융합은 상보 필터의 사용, 예컨대, 센서 기반 방위각

에 HPF를 적용하는 것 및 영상 기반 방위각

에 LPF를 적용하는 것을 포함할 수 있다.In some example implementations, as represented by block 710 , processing unit 150 is a sensor-based azimuth

and image-based azimuth

Filtered azimuth by fusing

can be calculated. In certain instances, such fusion may be achieved by use of a complementary filter, such as sensor-based azimuth.

Applying HPF to and image-based azimuth

may include applying LPF to

및 필터링된 방위각

의 차이

를 계산할 수 있다.In some example implementations, as represented by block 720 , the processing unit 150 is a sensor-based azimuth

and filtered azimuth

difference of

Can be calculated.

에 오차 보상 값

을 누적하는 방식으로 오차 보상 값

을 갱신할 수 있다. 예를 들어, 갱신된 오차 보상 값

은 UAV(10)의 차후 영상 기반 방위각

이 산출될 때까지 UAV(10)의 오차 보상된 센서 기반 방위각

을 산출하는 데에 사용될 수 있고, 그러한 경우에 만약 오차 보상 값

이 누적되지 않는다면 이전에 보상된 오차만큼 오차가 존재할 것임을 감안하여, 오차 보상 값

은 누적식으로 갱신될 수 있다.In some example implementations, as represented by block 730 , the processing unit 150 is configured to calculate the difference

error compensation value in

error compensation value by accumulating

can be updated. For example, the updated error compensation value

is the subsequent image-based azimuth of the UAV (10).

The error-compensated sensor-based azimuth of the UAV 10 until

can be used to calculate , in which case if the error compensation value

If this is not accumulated, considering that there will be an error equal to the previously compensated error, the error compensation value

can be updated cumulatively.

Also with reference to FIG. 8 , another exemplary flow of compensation for errors in azimuth estimation of the UAV 10 is discussed. Blocks 810 to 840 of FIG. 8 may be included in block 245 of FIG. 2 .

및 영상 기반 방위각

의 차이

를 계산할 수 있다.In some example implementations, as represented by block 810 , processing unit 150 is a sensor-based azimuth

and image-based azimuth

difference of

Can be calculated.

가 임계 값(가령, 30도) 이상인지를 판정할 수 있다.In some example implementations, as represented by block 820 , the processing unit 150 is configured to calculate the difference

It may be determined whether is greater than or equal to a threshold (eg, 30 degrees).

가 임계 값 미만인 경우, 처리 유닛(150)은 계산된 차이

에 오차 보상 값

을 누적하는 방식으로 오차 보상 값

을 갱신할 수 있다. 예를 들어, 갱신된 오차 보상 값

은 UAV(10)의 차후 영상 기반 방위각

이 산출될 때까지 UAV(10)의 오차 보상된 센서 기반 방위각

을 산출하는 데에 사용될 수 있고, 그러한 경우에 만약 오차 보상 값

이 누적되지 않는다면 이전에 보상된 오차만큼 오차가 존재할 것임을 감안하여, 오차 보상 값

은 누적식으로 갱신될 수 있다.As indicated by block 830 , if the difference

is less than the threshold, the processing unit 150 calculates the difference

error compensation value in

error compensation value by accumulating

can be updated. For example, the updated error compensation value

is the subsequent image-based azimuth of the UAV (10).

The error-compensated sensor-based azimuth of the UAV 10 until

can be used to calculate , in which case if the error compensation value

If this is not accumulated, considering that there will be an error equal to the previously compensated error, the error compensation value

can be updated cumulatively.

가 임계 값 이상인 경우, 처리 유닛(150)은 오차 보상 값

을 초기 값으로 초기화할 수 있다. 환언하면, 전술된 바와 같이 누적식으로 오차 보상 값

을 갱신하는 것은 차이

가 어느 정도 한도 내에 있을 때에 유용하다. 만일 차이

에 갑작스런 큰 변화가 있다면, 이 변화는 지자기 외란보다는 실내 환경 내의 사전결정된 특징에 포함된 인접한 부분들 간의 상당한 특징 차이(가령, 두 인접한 직선 부분의 절대 방위각 간의 큰 차이)에 기인할 공산이 높다. 이에 따라, 몇몇 예에서, 처리 유닛(150)은 그러한 경우에, 오차 보상 값

을 계속해서 누적시키는 것 대신에, 오차 보상 값

을 초기화할 수 있다.As indicated by block 840 , if the difference

is greater than or equal to the threshold value, the processing unit 150 sets the error compensation value

can be initialized to an initial value. In other words, as described above, the error compensation value is cumulative

Updating is the difference

It is useful when is within a certain limit. if the difference

If there is a sudden large change in , this change is more likely to be due to significant feature differences between adjacent parts included in the predetermined feature in the indoor environment (eg, large difference between the absolute azimuths of two adjacent straight-line parts) rather than geomagnetic disturbances. Accordingly, in some examples, the processing unit 150 may, in such a case, determine the error compensation value.

Instead of continuously accumulating

can be initialized.

이) 임계 시간(가령, 3초) 이상 수신되지 않는 경우에, 처리 유닛(150)은 오차 보상 값

을 초기 값으로 초기화할 수 있다. 이 경우에도, 실내 환경의 이전에 검출된 특징을 기반으로 설정된 오차 보상 값

을 계속해서 누적시키는 것은 오히려 센서 기반 방위각

의 오차를 늘릴 수 있으므로, 처리 유닛(150)은 이를 방지하기 위해 오차 보상 값

을 초기화할 수 있다.Additionally, whether in the error compensation of FIG. 7 or the error compensation of FIG. 8 , the image data from the imaging unit 140 (and thus the image-based azimuth as a result of the image processing 240 )

If this) is not received for more than a threshold time (eg, 3 seconds), the processing unit 150 sets the error compensation value

can be initialized to an initial value. Even in this case, the error compensation value set based on previously detected features of the indoor environment

It is rather the sensor-based azimuth to continuously accumulate

Since it is possible to increase the error of the processing unit 150, the error compensation value to prevent this

can be initialized.

9 is a flow diagram illustrating an example of a process 900 for estimating the azimuth of the UAV 10 . As noted above, the UAV 10 may operate in an indoor environment that includes predetermined features (eg, artifacts). For example, the process 900 of FIG. 9 may be performed by the navigation system 100 (specifically, the processing unit 150 ) of the UAV 10 . Also, as an example, process 900 of FIG. 9 may be performed without the use of assistance information from an external navigation system. Other example flows of process 900 are also contemplated. For example, process 900 may also include additional operations not shown and/or may include some but not all of the operations illustrated in FIG. 9 .

In operation 910 , an azimuth of the UAV 10 is calculated based on the magnetic field measurement data received from the geomagnetic sensor 130 of the UAV 10 . As described above, the magnetic field measurement data may be used to correct a divergence error occurring in calculating an azimuth angle based on the measurement data received from the gyroscope 110 . In some examples, data may be collected from the two-dimensional in-plane rotation of the geomagnetic sensor 130 during initialization of the UAV 10 , and calibration of the geomagnetic sensor 130 may be performed based on the collected data. Then, the azimuth of the UAV 10 may be calculated based on the magnetic field measurement data to which the calibration is applied.

At operation 920 , a sensor-based azimuth is determined from the calculated azimuth of the UAV 10 and a preset error compensation value. For example, after the error compensation value is initialized, it may be updated as described below.

In operation 930 , a predetermined characteristic of the indoor environment is detected based on the image data received from the imaging unit 140 of the UAV 10 . A period in which image data is received from the imaging unit 140 may be greater than a period in which magnetic field measurement data is received from the geomagnetic sensor 130 . In a particular example, the predetermined feature may include a straight portion having a predetermined absolute azimuth (eg, with respect to magnetic north).

At operation 940 , an image-based azimuth is determined from the detected predetermined feature. In a particular example, the image-based azimuth can be calculated from the relative angle of the straight-line portion of the predetermined feature with respect to the UAV 10 and the predetermined absolute azimuth of the straight-line portion.

In operation 950 , the error compensation value is updated based on the sensor-based azimuth and the image-based azimuth. For example, the sensor-based azimuth and the image-based azimuth may be processed as the filtered azimuth. Such processing may include filtering the sensor-based azimuth and the image-based azimuth with a complementary filter to yield a filtered azimuth. Then, the error compensation value may be set as a value in which the error compensation value is accumulated in the difference between the sensor-based azimuth and the filtered azimuth. In another example, based on whether the difference between the sensor-based azimuth and the image-based azimuth is equal to or greater than a threshold value, the error compensation value may be set as an initial value or a value in which the difference is accumulated in the error compensation value. Also, in any example, when image data is not received from the imaging unit 140 for more than a threshold time, the error compensation value may be set to an initial value.

For further details of the process 900 of FIG. 9 , reference may be made to the description above regarding the navigation system 100 of the UAV 10 .

In certain instances, any suitable type of apparatus, device, system, machine, etc. referred to herein may be, include, or be embodied in a computing device. A computing device may include a processor and a computer-readable storage medium readable by the processor. The processor may execute one or more instructions stored in a computer-readable storage medium. The processor may also read other information stored within the computer readable storage medium. Additionally, the processor may store new information in the computer-readable storage medium and update any information stored in the computer-readable storage medium. The processor is, for example, a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a processor core, a microprocessor. , a microcontroller, a Field-Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), other hardware and logic circuitry, or any suitable combination thereof. can A computer-readable storage medium is encoded with various information, such as a set of processor executable instructions and/or other information that can be executed by a processor. For example, a computer-readable storage medium may include computer program instructions that, when executed by a processor, cause a computing device (eg, a processor) to perform some operations disclosed herein and/or information, data, and/or information used in those operations; Variables, constants, data structures, etc. can be stored inside. Computer-readable storage media include, for example, read-only memory (ROM), random-access memory (RAM), volatile memory, non-volatile memory, removable Removable memory, non-removable memory, flash memory, solid-state memory, other types of memory devices, magnetic media such as hard disks, floppy disks and magnetic tapes, CDs - optical recording media such as ROM, DVD, magneto-optical media such as floppy disks, other types of storage devices and storage media, or any suitable combination thereof.

In certain instances, the acts, techniques, processes, or any aspect or portion thereof described herein may be embodied in a computer program product. Such computer programs can be implemented in any type (eg, compiled or interpreted) programming language that can be executed by a computer, such as assembly, machine language, procedural. It may be implemented in a language, an object-oriented language, etc., and may be combined with a hardware implementation. The computer program product may be distributed in the form of a computer-readable storage medium or may be distributed online. For online distribution, some or all of the computer program product may be temporarily stored or temporarily created in a server (eg, a computer-readable storage medium of the server).

The above description has been presented to illustrate and describe some examples in detail. It will be appreciated by those skilled in the art that many modifications and variations are possible in light of the above teachings without departing from the scope of the present disclosure. In various instances, the techniques described above are performed in a different order, and/or some of the components of the systems, architectures, devices, circuits and the like described above are combined or combined in different ways, replaced by other components or equivalents thereof, or Even with substitution, appropriate results can be achieved.

Therefore, the scope of the present disclosure should not be limited to the form disclosed, but should be defined by the following claims and their equivalents.

10: unmanned aerial vehicle
100: navigation system
110: gyroscope
120: accelerometer
130: geomagnetic sensor
140: imaging unit
150: processing unit

Claims (16)

As a method of estimating the azimuth of an unmanned aerial vehicle (UAV) operating in an indoor environment,
calculating the azimuth based on magnetic field measurement data received from a geomagnetic sensor of the UAV;
determining a sensor-based azimuth from the calculated azimuth and a preset error compensation value;
detecting a predetermined characteristic of the indoor environment based on image data received from an imaging unit of the UAV;
determining an image-based azimuth from the detected predetermined feature;
updating the error compensation value based on the sensor-based azimuth and the image-based azimuth,
wherein the predetermined feature comprises a straight-line portion having a predetermined absolute azimuth, and wherein determining the image-based azimuth includes calculating the image-based azimuth from the absolute azimuth and a relative angle of the straight-line portion with respect to the UAV. containing,
Way. delete The method of claim 1,
The updating of the error compensation value includes calculating a filtered azimuth by filtering the sensor-based azimuth and the image-based azimuth with a complementary filter, and adding the difference between the sensor-based azimuth and the filtered azimuth. Comprising the step of setting the error compensation value to the accumulated value of the error compensation value,
Way. The method of claim 1,
The updating of the error compensation value includes using the error compensation value as an initial value or an accumulated value of the error compensation value based on whether the difference between the sensor-based azimuth and the image-based azimuth is equal to or greater than a threshold value. comprising the steps of setting
Way. The method of claim 1,
Setting the error compensation value to an initial value when the image data is not received from the imaging unit for more than a threshold time,
Way. The method of claim 1,
Further comprising the step of performing calibration (calibration) of the geomagnetic sensor based on data collected from the two-dimensional plane rotation of the geomagnetic sensor, and calculating the azimuth is based on the magnetic field measurement data to which the calibration is applied Comprising the step of calculating the azimuth with
Way. The method of claim 1,
Calculating the azimuth includes calculating the azimuth as a gyroscope-based azimuth based on angular velocity measurement data received from a gyroscope of the UAV, and determining the azimuth based on the magnetic field measurement data based on a geomagnetic sensor Comprising the steps of calculating as an azimuth, and calculating the azimuth by filtering the gyroscope-based azimuth and the geomagnetic sensor-based azimuth with a complementary filter,
Way. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a computer processor, cause the computer processor to perform the method according to any one of claims 1 and 3-7. A computing device comprising:
processor and
A memory comprising: a memory encoded as a set of computer program instructions executable by the processor for estimating an azimuth of an Unmanned Aerial Vehicle (UAV) operating in an indoor environment, the set comprising:
a command for calculating the azimuth of the UAV based on the magnetic field measurement data received from the geomagnetic sensor of the UAV;
instructions for determining a sensor-based azimuth from the calculated azimuth and a preset error compensation value;
instructions for detecting a predetermined characteristic of the indoor environment based on image data received from the imaging unit of the UAV;
instructions for determining an image-based azimuth from the detected predetermined feature;
a command for updating the error compensation value based on the sensor-based azimuth and the image-based azimuth,
The predetermined feature includes a straight line portion having a predetermined absolute azimuth, and the instructions for determining the image-based azimuth include instructions for calculating the image-based azimuth from the absolute azimuth and a relative angle of the linear portion with respect to the UAV. containing,
computing device. delete The method of claim 9,
The command for updating the error compensation value includes a command for calculating a filtered azimuth by filtering the sensor-based azimuth and the image-based azimuth with a complementary filter, and adding the error compensation value to the difference between the sensor-based azimuth and the filtered azimuth. including a command for setting the error compensation value to an accumulated value,
computing device. The method of claim 9,
The command for updating the error compensation value is based on whether the difference between the sensor-based azimuth and the image-based azimuth is equal to or greater than a threshold value, and sets the error compensation value as an initial value or a value obtained by adding the error compensation value to the difference. containing the command to set,
computing device. The method of claim 9,
The set further includes instructions for setting the error compensation value to an initial value when the image data is not received from the imaging unit for more than a threshold time,
computing device. The method of claim 9,
The set further includes instructions for performing calibration of the geomagnetic sensor based on data collected from the two-dimensional rotation of the geomagnetic sensor, and the instruction for calculating the azimuth is the magnetic field measurement data to which the calibration is applied. Comprising a command to calculate the azimuth based on,
computing device. The method of claim 9,
The command for calculating the azimuth is a command for calculating the azimuth as a gyroscope-based azimuth based on angular velocity measurement data received from the gyroscope of the UAV, and calculating the azimuth as a geomagnetic sensor-based azimuth based on the magnetic field measurement data and a command for calculating the azimuth by filtering the gyroscope-based azimuth and the geomagnetic sensor-based azimuth with a complementary filter.
computing device. An unmanned aerial vehicle comprising the geomagnetic sensor according to any one of claims 9 and 11 to 15, an imaging unit, and a computing device.

Images