WO2023184487A1 - 无人机避障方法、装置、无人机、遥控设备和存储介质 - Google Patents
无人机避障方法、装置、无人机、遥控设备和存储介质 Download PDFInfo
- Publication number
- WO2023184487A1 WO2023184487A1 PCT/CN2022/084823 CN2022084823W WO2023184487A1 WO 2023184487 A1 WO2023184487 A1 WO 2023184487A1 CN 2022084823 W CN2022084823 W CN 2022084823W WO 2023184487 A1 WO2023184487 A1 WO 2023184487A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- obstacle avoidance
- image area
- mark
- obstacle
- image
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000007613 environmental effect Effects 0.000 claims description 29
- 238000001514 detection method Methods 0.000 claims description 10
- 230000009194 climbing Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
Definitions
- the present invention relates to the technical field of UAV control, and in particular to a UAV obstacle avoidance method, device, UAV, remote control equipment and storage medium.
- UAV obstacle avoidance mainly includes UAV automatic obstacle avoidance and user manual operation of UAV obstacle avoidance.
- UAV automatic obstacle avoidance means that the UAV avoids obstacles according to presets after detecting obstacles. Obstacle avoidance is carried out through obstacle avoidance methods.
- the user's manual operation of the drone to avoid obstacles refers to the user of the drone controlling the drone to avoid obstacles by controlling the remote control device.
- Embodiments of the present invention provide a UAV obstacle avoidance method, device, UAV, remote control device and storage medium to improve the user experience when the UAV avoids obstacles, thereby improving the safety of the UAV.
- embodiments of the present invention provide a UAV obstacle avoidance method, which method includes:
- a first mark is superimposed on a first image area of the real-time environment image, and a second mark is superimposed on a second image area of the real-time environment image.
- the first image area is different from the second image area.
- the first mark is different from said second mark;
- the first mark and the second mark are used to indicate that the environment space corresponding to the first image area and the environment space corresponding to the second image area have different obstacle avoidance costs, and the obstacle avoidance cost is the same as Relevant by safety, power consumed by obstacle avoidance, and/or length of obstacle avoidance path.
- embodiments of the present invention provide a drone obstacle avoidance device, including a memory and a processor; wherein, executable code is stored on the memory, and when the executable code is executed by the processor, Cause said processor to implement:
- a first mark is superimposed on a first image area of the real-time environment image, and a second mark is superimposed on a second image area of the real-time environment image.
- the first image area is different from the second image area.
- the first mark is different from said second mark;
- the first mark and the second mark are used to indicate that the environment space corresponding to the first image area and the environment space corresponding to the second image area have different obstacle avoidance costs, and the obstacle avoidance cost is the same as Relevant by safety, power consumed by obstacle avoidance, and/or length of obstacle avoidance path.
- an embodiment of the present invention provides a drone, including the obstacle avoidance device for a drone provided in the second aspect of the embodiment of the present invention.
- an embodiment of the present invention provides a remote control device for a drone, including the obstacle avoidance device for a drone provided in the second aspect of the embodiment of the present invention.
- embodiments of the present invention provide a computer-readable storage medium.
- Program instructions are stored in the computer-readable storage medium.
- the program instructions are used to implement the UAV provided in the first aspect of the embodiment of the present invention. Obstacle avoidance methods.
- different marks can be superimposed on different image areas in the real-time environment image to indicate the obstacle avoidance cost of the environmental space corresponding to the image area.
- users can more intuitively understand the obstacle avoidance logic of drones, predict the obstacle avoidance behavior of drones, and reduce the occurrence of misoperations.
- it is helpful to assist users in performing safer and more efficient obstacle avoidance control operations.
- Figure 1 is a schematic flow chart of a UAV obstacle avoidance method provided by an embodiment of the present invention
- Figure 2 is a schematic diagram of the division of different image areas provided by an embodiment of the present invention.
- Figure 3 is a schematic diagram of the principle of constructing an obstacle avoidance cost map provided by an embodiment of the present invention.
- Figure 4 is a schematic diagram of an automatic obstacle avoidance path projected onto a real-time environment image according to an embodiment of the present invention
- Figure 5 is a schematic diagram of superimposing corresponding prompts in a real-time environment image provided by an embodiment of the present invention
- FIG. 6 is another schematic diagram of superimposing corresponding prompts in a real-time environment image provided by an embodiment of the present invention.
- FIG. 7 is another schematic diagram of superimposing corresponding prompts in a real-time environment image provided by an embodiment of the present invention.
- Figure 8 is a schematic structural diagram of a UAV obstacle avoidance device provided by an embodiment of the present invention.
- the words “if” or “if” as used herein may be interpreted as “when” or “when” or “in response to determination” or “in response to detection.”
- the phrase “if determined” or “if (stated condition or event) is detected” may be interpreted as “when determined” or “in response to determining” or “when (stated condition or event) is detected )” or “in response to detecting (a stated condition or event)”.
- FIG. 1 is a flow chart of a UAV obstacle avoidance method provided by an embodiment of the present invention. This method can be applied to UAVs or remote control devices that interact with UAVs. As shown in Figure 1, the method includes the following steps:
- the first mark and the second mark are used to indicate that the environment space corresponding to the first image area and the environment space corresponding to the second image area have different obstacle avoidance costs.
- the obstacle avoidance cost is related to the safety, the power consumed by obstacle avoidance and the /or related to the length of the obstacle avoidance path.
- real-time environment images can be captured through the image capturing device set up on the UAV, and the first image area and the second image area in the real-time environment image are determined.
- the two image areas are different. , they each correspond to different environmental spaces and the obstacle avoidance costs corresponding to each environmental space are different.
- the above obstacle avoidance cost is related to factors such as safety of passage, power consumed by obstacle avoidance, and the length of the obstacle avoidance path.
- the obstacle avoidance cost can be used to evaluate the advantages and disadvantages of UAVs in avoiding obstacles from different environmental spaces.
- the obstacle avoidance cost may be associated with one or more of pass safety, power consumed by obstacle avoidance, and the length of the obstacle avoidance path.
- the obstacle avoidance cost may be the sum of the multiple costs, and each of the multiple costs may be associated with one factor.
- the first cost is associated with passing safety
- the second cost is associated with the power consumed by obstacle avoidance
- the third cost is associated with the length of the obstacle avoidance path.
- the relationship between the obstacle avoidance cost and the pass safety, the power consumed by obstacle avoidance, and the length of the obstacle avoidance path can be expressed as:
- passing safety the worse the passing safety, the greater the cost of obstacle avoidance. For example, it is generally believed that the environmental space closer to the ground has more obstacles, and the environmental space higher in altitude has fewer obstacles. Then the safety of bypassing the obstacle from above is higher than the safety of bypassing the obstacle from below. Therefore, the environmental space below the obstacle is less safe and the corresponding obstacle avoidance cost is higher.
- passing safety can also be associated with various factors such as the detection range of the obstacle detection device, the density of obstacles, and the distance to the obstacles.
- passing through unknown environmental areas is less safe and the cost of obstacle avoidance is higher; passing through environmental areas with dense obstacles is less safe and the cost of obstacle avoidance is higher; distance barriers It is less safe to pass through environmental areas within the preset distance of objects, and the cost of obstacle avoidance is high.
- the greater the power consumed by obstacle avoidance the greater the cost of obstacle avoidance.
- the current relative position between the environment space and the drone will affect the power consumed by obstacle avoidance.
- the power consumed by the drone climbing upward is greater than that consumed by the drone flying left or right.
- the power consumed by the drone flying to the left or right is relatively low, so the obstacle avoidance cost corresponding to this type of environmental space is low, and the power consumed by the drone during its upward climb is relatively high, then this type of environmental space consumes relatively high power.
- the obstacle avoidance cost corresponding to the environmental space is higher.
- the longer the obstacle avoidance path the greater the cost of obstacle avoidance. It is understandable that when the length of the obstacle avoidance path is longer, the power consumed by the drone and the time spent flying are longer. This is not expected, so the corresponding obstacle avoidance cost is higher.
- the first image area is located above the obstacle image area
- the second image area is located above the obstacle image area. both sides. It can be understood that when the path length is the same, the power consumed by the drone during its upward climb is greater than the power consumed by the drone flying to both sides. Therefore, the image area above the obstacle image area corresponds to the image area on both sides.
- the environmental space has different obstacle avoidance costs.
- the first image area is located above the obstacle image area, and the second image area is located below the obstacle image area. It can be understood that since the safety of passing a drone flying downward is different from that of climbing upward, the environmental space corresponding to the image area above the obstacle image area and the image area below has different obstacle avoidance costs.
- the first image area and the second image area can be marked with different marks, so that the user can intuitively understand the corresponding advantages and disadvantages of the drone through different environmental spaces.
- the above-mentioned marks can be one or more of color marks, text marks, and texture marks, or other marking methods can be used to mark different image areas, so that users can intuitively understand that different image areas correspond to The advantages and disadvantages of the environmental space.
- the first image area is marked with a green color with a certain degree of transparency
- the second image area is marked with a yellow color with the same degree of transparency.
- the first image area is highlighted and the second image area is displayed with a certain grayscale, etc.
- the method provided by the present invention is executed in a drone, after the drone obtains the real-time environment image through the image shooting device, different markers can be directly superimposed on different image areas of the real-time environment image, and then the different markers will be superimposed
- the real-time environment image is sent to the remote control device, and the remote control device displays the real-time environment image superimposed with different marks through the display device.
- the drone can send the real-time environment image to the remote control device.
- the remote control device acquires a real-time environment image, then superimposes different marks on different image areas of the real-time environment image, and finally displays the real-time environment image superimposed with different marks through the display device.
- the process of superimposing the first mark on the first image area of the real-time environment image and superposing the second mark on the second image area of the real-time environment image can be implemented as: according to the obstacle avoidance cost map, on the first image area of the real-time environment image.
- the first mark is superimposed on the image area
- the second mark is superimposed on the second image area of the real-time environment image.
- the obstacle avoidance cost map is obtained based on the data detected by the drone's detection device.
- the above-mentioned detection device can be an image capturing device, a TOF (Time of Flight) sensor, an ultrasonic sensor, or a lidar sensor.
- the image capturing device includes but is not limited to a monocular camera, a binocular camera, etc.
- an obstacle avoidance cost map can be generated, and different markers are superimposed on different image areas in the real-time environment image based on the obstacle avoidance cost map.
- the above obstacle avoidance cost map can be composed of multiple grids, each grid corresponds to a certain unit space in the actual environment, and each grid corresponds to its own obstacle avoidance cost.
- the obstacle avoidance cost corresponding to each grid is related to factors such as passing safety, the power consumed by obstacle avoidance, and/or the length of the obstacle avoidance path.
- the point cloud detected by the detection device can be projected into the body coordinate system of the drone, and then discretized in the direction perpendicular to the depth.
- the five-pointed star in the figure represents the UAV
- the cuboid represents the obstacle in front of the UAV
- the point that the UAV is facing on the obstacle along the flight direction is O.
- flying the same distance the power consumed by the drone rising is higher than that consumed by flying to the left and right sides.
- the risk of encountering obstacles when the drone descends is high, and the safety of passing is low. It can be set that starting from point O, the obstacle avoidance cost of shifting one grid to the left and right is minimum, the obstacle avoidance cost of shifting upward by one grid is second, and the obstacle avoidance cost of shifting downward by one grid is infinite.
- superimposing the first mark on the first image area of the real-time environment image according to the obstacle avoidance cost map, and superimposing the second mark on the second image area of the real-time environment image may include: projecting the obstacle avoidance cost map onto the real-time environment image. , based on the obstacle avoidance cost corresponding to each grid in the projected obstacle avoidance cost map, a first mark is superimposed on the first image area of the real-time environment image, and a second mark is superimposed on the second image area of the real-time environment image.
- the image areas corresponding to the obstacle avoidance cost grids corresponding to different values or different value ranges in the real-time environment image can be determined as different image areas. , and then overlay corresponding markers for different image areas.
- the obstacle avoidance cost map can be constructed using the UAV body coordinate system as a reference.
- the obstacle avoidance cost map in the body coordinate system needs to be projected into the image coordinate system corresponding to the real-time environment image.
- the image capturing device is fixedly installed on the drone, the position and posture of the image capturing device relative to the drone is fixed during the flight of the drone, that is, the distance between the body coordinate system and the image coordinate system is If the rotation and translation relationships are fixed, projection can be performed based on the preset rotation and translation relationships.
- the image shooting device is installed on the UAV through a rotating device (such as a gimbal). During the flight of the UAV, the rotation and translation relationship between the body coordinate system and the image coordinate system changes. It can be based on the image.
- the real-time pose of the shooting device determines the rotation and translation relationship between the body coordinate system and the image coordinate system, and then projection is performed based on the determined rotation and translation relationship.
- the obstacle avoidance cost map can also be constructed using the world coordinate system as a reference.
- the obstacle avoidance cost map in the world coordinate system needs to be projected into the image coordinate system corresponding to the real-time environment image.
- the rotation and translation relationships between the world coordinate system and the image coordinate system change.
- the rotation and translation between the world coordinate system and the image coordinate system can be determined based on the real-time poses of the drone and the image shooting device.
- the translation relationship, or the rotation and translation relationship between the world coordinate system and the image coordinate system is determined based on the real-time pose of the drone and the real-time pose of the image shooting device relative to the drone, and then based on the determined rotation and translation relationship projection.
- the obstacle avoidance cost map can also be constructed using other coordinate systems as a reference.
- the projection can be realized based on the transformation relationship of the image coordinate system corresponding to the real-time environment image of this coordinate system, which will not be described again here.
- the above projection process can be performed by a drone or a remote control device. If executed by a drone, the drone can directly project the obstacle avoidance cost map into the real-time environment image based on the projection parameters required for projection. If it is executed by a remote control device, the drone can send the projection parameters required for projection to the remote control device, and the remote control device projects the obstacle avoidance cost map into the real-time environment image based on the projection parameters required for projection.
- the above-mentioned projection parameters may refer to the parameters required for the above-mentioned coordinate system transformation.
- an automatic obstacle avoidance path can be planned based on the obstacle avoidance cost map.
- the obstacle avoidance costs corresponding to multiple grids in the obstacle avoidance cost map can be substituted into the mathematical model, and the loss values calculated by the combination of multiple grids can be continuously optimized through the mathematical model to find the optimal solution.
- the path formed by connecting the corresponding spatial positions of the multiple grids corresponding to the optimal solution in the actual environment is the automatic obstacle avoidance path.
- the distance the drone needs to climb upward to bypass the obstacle is d1
- the distance the drone needs to fly to bypass the obstacle from the right is d2
- the distance the drone needs to fly to bypass the obstacle from the left is d2.
- the distance required to fly around the obstacle is d3, and d1 is smaller than d2 and smaller than d3.
- the UAV will choose to climb upward to avoid obstacles.
- the obstacle avoidance cost map the obstacle avoidance cost of one grid offset to the left and right is the smallest, the obstacle avoidance cost of one grid offset upward is second, and the obstacle avoidance cost of a downward offset is the second.
- the obstacle avoidance cost of a grid is infinite. In this way, even if the upward flight distance is the shortest, it will not necessarily be used as an automatic obstacle avoidance path. Instead, the power consumption of obstacle avoidance and the safety of passage will be taken into consideration.
- the automatic obstacle avoidance path can be projected onto the real-time environment image, and the projected real-time image can be displayed on the remote control device.
- environmental images As shown in Figure 4, it is a schematic diagram of displaying the automatic obstacle avoidance path in the real-time environment image through projection.
- the five-pointed stars in the picture are used to indicate the spatial positions that the drone will reach one after another.
- the line formed by multiple five-pointed stars represents the automatic obstacle avoidance path projected into the real-time environment image.
- the direction of the arrow is used to indicate the flight direction of the drone. .
- the process of projecting the automatic obstacle avoidance path to the real-time environment image is similar to the process of projecting the obstacle avoidance cost map to the real-time environment image.
- the process of projecting the automatic obstacle avoidance path will not be elaborated here.
- the automatic obstacle avoidance path is projected.
- the process can refer to the process of projecting the obstacle avoidance cost map described previously.
- the process of projecting the automatic obstacle avoidance path to the real-time environment image can be implemented by a drone, or it can also be implemented by a remote control device. If it is implemented by a remote control device, the drone can send the projection parameters required for projection to the remote control device, and the remote control device projects the automatic obstacle avoidance path into the real-time environment image based on the projection parameters required for projection.
- the projection parameters required for projection include the parameters required for the aforementioned coordinate conversion, such as the posture of the drone when capturing real-time environmental images, the posture of the image capturing device, the posture of the image capturing device relative to the drone, the cloud One or more of the position and posture of the platform relative to the drone and the position and posture of the image capturing device relative to the platform.
- the above-mentioned first preset condition may be, for example, a first distance range interval.
- the operation of overlaying the first mark on the first image area of the real-time environment image, and overlaying the second mark on the second image area of the real-time environment image may be performed.
- the process of projecting the automatic obstacle avoidance path to the real-time environment image can also be triggered when the distance between the drone and the obstacle meets the first preset condition.
- the projected automatic obstacle avoidance path can be superimposed on the real-time environment image for the user to view.
- prompt information can also be output to remind the user that the drone is performing automatic obstacle avoidance operations according to the automatic obstacle avoidance path. This allows the user to perceive that the current working status of the drone is performing automatic obstacle avoidance operations, and that the drone will perform automatic obstacle avoidance operations by overlaying and displaying the projected automatic obstacle avoidance path on the real-time environment image.
- a prompt message is output to remind the user that the drone is too close to the obstacle, and the drone will perform deceleration and/or climbing operations.
- the second preset condition may be, for example, a second distance range interval, and the maximum value of the second distance range interval is smaller than the minimum value of the first distance range interval.
- a prompt message can be output to remind the user that the drone is too close to the obstacle, and the drone will perform deceleration and/or climbing operations.
- the drone can directly choose to avoid obstacles by decelerating and climbing.
- a corresponding as shown in the figure can be added to the upper right corner of the real-time environment image.
- Icon which can be a warning sign to remind the user that the drone is too close to an obstacle.
- a prompt message is output to remind the user that a distant obstacle is detected and that it is safe to go straight.
- the above-mentioned third preset condition may be, for example, a third distance range interval, and the minimum value of the third distance range interval is greater than the maximum value of the aforementioned first distance range interval.
- the distance between the drone and the obstacle is in the third distance range, it means that the drone is relatively far away from the obstacle, and corresponding prompt information can be output to remind the user that a long-distance obstacle has been detected and that it is safe to go straight.
- different markers can be superimposed on different image areas in the real-time environment image, and the environmental spaces corresponding to different image areas have different obstacle avoidance costs.
- users can understand the obstacle avoidance logic of drones more intuitively by superimposing different marks on different image areas.
- different marks superimposed on different image areas are helpful to assist users in performing safer and more efficient obstacle avoidance control operations.
- FIG. 8 Yet another exemplary embodiment of the present invention provides an obstacle avoidance device for a UAV, as shown in Figure 8.
- the device includes:
- Memory 1910 for storing computer programs
- Processor 1920 used to run the computer program stored in memory 1910 to implement:
- a first mark is superimposed on a first image area of the real-time environment image, and a second mark is superimposed on a second image area of the real-time environment image.
- the first image area is different from the second image area.
- the first mark is different from said second mark;
- the first mark and the second mark are used to indicate that the environment space corresponding to the first image area and the environment space corresponding to the second image area have different obstacle avoidance costs, and the obstacle avoidance cost is the same as Relevant by safety, power consumed by obstacle avoidance, and/or length of obstacle avoidance path.
- the first image area is located above the obstacle image area, and the second image area is located on both sides of the obstacle image area.
- the first image area is located above the obstacle image area, and the second image area is located below the obstacle image area.
- the mark is one or more of color marks, text marks, and texture marks.
- superimposing a first mark on the first image area of the real-time environment image, and superimposing a second mark on the second image area of the real-time environment image includes:
- a first mark is superimposed on the first image area of the real-time environment image, and a second mark is superimposed on the second image area of the real-time environment image.
- the obstacle avoidance cost map is based on the unmanned The data detected by the machine's detection device is obtained.
- the automatic obstacle avoidance path of the UAV is planned based on the obstacle avoidance cost map.
- processor 1920 is used to:
- processor 1920 is also used to:
- the automatic obstacle avoidance path of the drone is projected into the real-time environment image.
- processor 1920 is also used to:
- prompt information is output to remind the user that the drone is performing an automatic obstacle avoidance operation according to the automatic obstacle avoidance path.
- processor 1920 is also used to:
- prompt information is output to remind the user that the UAV is too close to the obstacle, and the UAV will perform deceleration and/or climbing operations.
- processor 1920 is also used to:
- a prompt message is output to remind the user that a long-distance obstacle is detected and it is safe to go straight.
- the UAV obstacle avoidance device shown in Figure 8 can perform the method of the embodiment shown in Figures 1 to 7.
- parts not described in detail in this embodiment please refer to the relevant description of the embodiment shown in Figures 1 to 7.
- the implementation process and technical effects of this technical solution please refer to the description in the embodiment shown in Figures 1 to 7, and will not be described again here.
- An embodiment of the present invention also provides a UAV, which may include the UAV obstacle avoidance device provided in the embodiment shown in FIG. 8 .
- Yet another embodiment of the present invention also provides a remote control device for a drone.
- the remote control device may include the obstacle avoidance device for the drone provided in the embodiment shown in FIG. 8 .
- embodiments of the present invention also provide a computer-readable storage medium in which executable code is stored, and the executable code is used to implement UAV obstacle avoidance as provided in the foregoing embodiments. method.
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
本发明实施例提供一种无人机避障方法、装置、无人机、遥控设备和存储介质，该方法包括：获取无人机的图像拍摄装置拍摄的实时环境图像；在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记；其中，第一标记和第二标记用于指示第一图像区域对应的环境空间和第二图像区域对应的环境空间具有不同的避障代价，避障代价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。采用本发明，可以在实时环境图像中的不同图像区域叠加不同的标记，不同图像区域对应的环境空间具有的避障代价不同。
Description
本发明涉及无人机控制技术领域，尤其涉及一种无人机避障方法、装置、无人机、遥控设备和存储介质。
相关技术中，无人机的避障主要包括无人机自动避障和用户手动操作无人机避障，无人机自动避障指的是无人机在检测到障碍物后按照预设避障方法进行避障，用户手动操作无人机避障指的是无人机的用户通过操控遥控设备控制无人机进行避障。
然而，目前避障时的提示较少。对于无人机自动避障的应用场景，用户不易得知无人机的自动避障逻辑，难以预料无人机的避障行为，有时会产生误操作；对于用户手动操作无人机避障的应用场景，需要用户有较高的无人机操作能力来应对各种类型的环境，对于新手用户来说不太友好。
发明内容
本发明实施例提供一种无人机避障方法、装置、无人机、遥控设备和存储介质，用以提高无人机避障时的用户体验，进而提高无人机的安全性。
第一方面，本发明实施例提供一种无人机避障方法，该方法包括：
获取无人机的图像拍摄装置拍摄的实时环境图像；
在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述第一图像区域不同于所述第二图像区域，所述第一标记不同于所述第二标记；
其中，所述第一标记和所述第二标记用于指示所述第一图像区域对应的环境空间和所述第二图像区域对应的环境空间具有不同的避障代价，所述避障代 价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。
第二方面，本发明实施例提供一种无人机避障装置，包括存储器、处理器；其中，所述存储器上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器实现：
获取无人机的图像拍摄装置拍摄的实时环境图像；
在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述第一图像区域不同于所述第二图像区域，所述第一标记不同于所述第二标记；
其中，所述第一标记和所述第二标记用于指示所述第一图像区域对应的环境空间和所述第二图像区域对应的环境空间具有不同的避障代价，所述避障代价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。
第三方面，本发明实施例提供一种无人机，包括本发明实施例第二方面提供的无人机避障装置。
第四方面，本发明实施例提供一种无人机的遥控设备，包括本发明实施例第二方面提供的无人机避障装置。
第五方面，本发明实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有程序指令，所述程序指令用于实现本发明实施例第一方面提供的无人机避障方法。
采用本发明，可以在实时环境图像中的不同图像区域叠加不同的标记，用以指示图像区域对应的环境空间的避障代价。在无人机自动避障的应用场景中，能够让用户更加直观地了解无人机的避障逻辑，预料无人机的避障行为，减少误操作的产生。在用户手动操作无人机避障的应用场景中，有利于辅助用户进行更加安全高效的避障控制操作。
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的 一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的一种无人机避障方法的流程示意图；
图2为本发明实施例提供的一种不同图像区域的划分示意图；
图3为本发明实施例提供的一种构建避障代价地图的原理示意图；
图4为本发明实施例提供的一种向实时环境图像投影自动避障路径的示意图；
图5为本发明实施例提供的一种在实时环境图像中叠加相应提示的示意图；
图6为本发明实施例提供的另一种在实时环境图像中叠加相应提示的示意图；
图7为本发明实施例提供的又一种在实时环境图像中叠加相应提示的示意图；
图8为本发明实施例提供的一种无人机避障装置的结构示意图。
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在本发明实施例中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本发明。在本发明实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义，“多种”一般包含至少两种。
取决于语境，如在此所使用的词语“如果”、“若”可以被解释成为“在……时”或“当……时”或“响应于确定”或“响应于检测”。类似地，取决于语境，短语“如果确定”或“如果检测(陈述的条件或事件)”可以被解释成为“当确 定时”或“响应于确定”或“当检测(陈述的条件或事件)时”或“响应于检测(陈述的条件或事件)”。
另外，下述各方法实施例中的步骤时序仅为一种举例，而非严格限定。
图1为本发明实施例提供的一种无人机避障方法的流程图，该方法可以应用于无人机或者与无人机交互的遥控设备。如图1所示，该方法包括如下步骤：
101、获取无人机的图像拍摄装置拍摄的实时环境图像。
102、在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记，第一图像区域不同于第二图像区域，第一标记不同于第二标记。
其中，第一标记和第二标记用于指示第一图像区域对应的环境空间和第二图像区域对应的环境空间具有不同的避障代价，避障代价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。
实际应用中，在无人机飞行的过程中，可以通过无人机设置的图像拍摄装置拍摄实时环境图像，确定实时环境图像中的第一图像区域以及第二图像区域，两种图像区域是不同的，它们各自对应着不同的环境空间且各环境空间对应的避障代价(cost)不同。
上述避障代价与通过安全性、避障消耗的电量、避障路径的长度等因素相关联，避障代价可以用于评价无人机从不同环境空间绕开障碍物的优劣性。
需要说明的是，避障代价可以与通过安全性、避障消耗的电量以及避障路径的长度中的一个或多个相关联。当避障代价与其中的多个相关联时，避障代价可以是多个代价之和，多个代价中的每一个可以与一种因素相关联。例如，第一代价与通过安全性相关联，第二代价与避障消耗的电量相关联，第三代价与避障路径的长度相关联。如此，在评价无人机从不同环境空间绕开障碍物的优劣性时综合考虑了多种因素，从而可以为更加安全高效的无人机避障提供支持。
可选地，避障代价与通过安全性、避障消耗的电量、避障路径的长度之间 的关系可以表示为：
通过安全性越差，避障代价越大；
避障消耗的电量越大，避障代价越大；
避障路径的长度越长，避障代价越大。
对于通过安全性来说，通过安全性越差避障代价越大。示例的，通常认为越靠近地面的环境空间存在的障碍物越多，越处于高空的环境空间存在的障碍物越少。那么从障碍物上方绕过障碍物的安全性相对于从障碍物下方绕过障碍物的安全性更高，因此障碍物下方的环境空间通过安全性较差，对应的避障代价较高。可选的，通过安全性还可以与障碍物检测装置的检测范围、障碍物的密集程度、以及与障碍物的距离等多种因素相关联。例如，未知的环境区域(障碍物检测装置未探测到的环境区域)通过安全性较差，避障代价较大；障碍物密集的环境区域通过安全性较差，避障代价较大；距离障碍物预设距离内的环境区域通过安全性较差，避障代价较大。
对于避障消耗的电量来说，避障消耗的电量越大避障代价越大。示例的，环境空间与无人机之间当前的相对位置会影响避障消耗的电量。具体的，对于移动相同的路径来说，无人机向上攀升消耗的电量要比无人机向左侧或右侧飞行消耗的电量要大。也就是说，无人机向左侧或者右侧飞行消耗的电量相对较低，那么此类环境空间对应的避障代价较低，无人机向上攀升过程消耗的电量相对较高，那么此类环境空间对应的避障代价较高。
对于避障路径的长度来说，避障路径的长度越长避障代价越大，可以理解的，当避障路径的长度越长时无人机飞行需要消耗的电量以及花销的时间都较多，这不是期望得到的，因此对应的避障代价较高。
基于上述避障代价与避障消耗的电量之间的关系，可选地，如图2所示，可以确定第一图像区域位于障碍物图像区域的上方，第二图像区域位于障碍物图像区域的两侧。可以理解的是，在路径长度相同的情况下，无人机向上攀升过程消耗的电量大于无人机向两侧飞行消耗的电量，所以障碍物图像区域上方的图像区域和两侧的图像区域对应的环境空间具有不同的避障代价。
基于上述避障代价与通过安全性之间的关系，可选地，可以确定第一图像区域位于障碍物图像区域的上方，第二图像区域位于障碍物图像区域的下方。可以理解的是，由于无人机向下飞行的通过安全性和向上攀升的通过安全性不同，所以障碍物图像区域上方的图像区域和下方的图像区域对应的环境空间具有不同的避障代价。
可以以不同标记对第一图像区域以及第二图像区域进行标注，这样可以直观上让用户清楚地了解到无人机通过不同的环境空间各自对应的优劣性能如何。
其中，上述标记可以是颜色标记、文字标记、纹理标记中的一个或多个，或者也可以采用其他的标记方式对不同图像区域进行标注，以让用户从直观上了解到通过不同的图像区域对应的环境空间的优劣性。
例如，将第一图像区域以某种程度的透明度叠加上绿色进行标注，将第二图像区域以同样程度的透明度叠加上黄色进行标注。再例如，在第一图像区域中标记上“这部分环境空间是易于无人机进行避障的空间”，在第二图像区域中标记上“不建议通过这部分环境空间进行避障”等字样。或者，再例如，第一图像区域进行高亮显示，第二图像区域以一定灰度进行显示等。
值得注意的是，本发明提供的方法可以在无人机中执行，也可以在遥控设备中执行：
如果本发明提供的方法在无人机中执行，则在无人机通过图像拍摄装置拍摄获得实时环境图像之后，可以直接在实时环境图像的不同图像区域叠加不同的标记，然后将叠加有不同标记的实时环境图像发送给遥控设备，遥控设备通过显示装置显示叠加有不同标记的实时环境图像。
如果本发明提供的方法在遥控设备中执行，则在无人机通过图像拍摄装置拍摄获得实时环境图像之后，无人机可以将实时环境图像发送给遥控设备。遥控设备获取实时环境图像，然后在实时环境图像的不同图像区域叠加不同的标记，最后通过显示装置显示叠加有不同标记的实时环境图像。
可选地，在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记的过程可以实现为：根据避障代价地图在实 时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记，避障代价地图是基于无人机的探测装置探测的数据得到的。
其中，上述探测装置可以是图像拍摄装置、TOF(Time of flight)传感器、超声波传感器、激光雷达传感器。其中，图像拍摄装置包括但不限于单目摄像头、双目摄像头等。
实际应用中，可以生成避障代价地图，基于避障代价地图在实时环境图像中的不同图像区域叠加不同标记。
其中，上述避障代价地图可以由多个栅格构成，各栅格与实际环境中的某个单位空间对应，每个栅格对应有各自的避障代价。各栅格对应的避障代价与通过安全性、避障消耗的电量和/或避障路径的长度等因素相关联。
下面简要介绍本申请一种实施例中构建避障代价地图的方法。
首先，可以将探测装置探测的点云投影到无人机的本体坐标系下，然后在垂直于深度的方向进行离散化。如图3所示，图中五角星表示无人机，长方体表示无人机前方的障碍物，无人机沿飞行方向在障碍物上正对着的点为O。考虑到飞行相同的距离，无人机上升所消耗的电量要比向左右侧飞行消耗的电量要高，无人机下降遇到障碍物的风险大，通过安全性低。可以设定从O点开始，左右偏移一个栅格的避障代价最小，向上偏移一个栅格的避障代价其次，向下偏移一个栅格的避障代价无穷大。
可选地，根据避障代价地图在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记可以包括：将避障代价地图投影到实时环境图像中，基于投影后的避障代价地图中各栅格对应的避障代价，在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记。
实际应用中，在将避障代价地图投影到实时环境图像之后，可以将对应于不同数值或者不同数值区间范围的避障代价的栅格在实时环境图像中对应的图像区域，确定为不同图像区域，进而为不同的图像区域叠加对应的标记。
下面简要介绍向实时环境图像投影避障代价地图的方案。
可选的，避障代价地图可以是以无人机本体坐标系作为参照构建的，在进行投影时，需要将本体坐标系下的避障代价地图投影到实时环境图像对应的图像坐标系中。如果图像拍摄装置是固定安装在无人机上的，在无人机飞行的过程中图像拍摄装置相对于无人机的位姿是固定不变的，也即本体坐标系到图像坐标系之间的旋转和平移关系固定不变，则可以基于预设的旋转和平移关系进行投影。而对于图像拍摄装置是通过转动装置(如云台)安装在无人机上的，在无人机飞行的过程中本体坐标系到图像坐标系之间的旋转和平移关系是变化的，可以基于图像拍摄装置的实时位姿确定本体坐标系到图像坐标系之间的旋转和平移关系，进而基于确定的旋转和平移关系进行投影。
可选的，避障代价地图还可以是以世界坐标系作为参照构建的，在进行投影时，需要将世界坐标系下的避障代价地图投影到实时环境图像对应的图像坐标系中，在无人机飞行的过程中世界坐标系到图像坐标系之间的旋转和平移关系是变化的，可以基于无人机和图像拍摄装置的实时位姿确定世界坐标系到图像坐标系之间的旋转和平移关系，或者基于无人机的实时位姿以及图像拍摄装置相对于无人机的实时位姿确定世界坐标系到图像坐标系之间的旋转和平移关系，进而基于确定的旋转和平移关系进行投影。
当然，避障代价地图也可以是以其他坐标系作为参照构建的，基于该坐标系实时环境图像对应的图像坐标系的转换关系即可实现投影，在此不做赘述。
另外值得注意的是，上述投影过程可以由无人机执行，也可以由遥控设备执行。如果是无人机执行，则无人机可以直接基于投影所需的投影参数将避障代价地图投影到实时环境图像中。如果是遥控设备执行，则无人机可以将投影所需的投影参数发送给遥控设备，由遥控设备基于投影所需的投影参数将避障代价地图投影到实时环境图像中。上述投影参数可以是指上述坐标系转换所需的参数。
在本发明实施例的另一方面，在获得避障代价地图之后，可以基于避障代价地图规划出自动避障路径。
具体来说，可以将避障代价地图中的多个栅格对应的避障代价代入到数学 模型，通过该数学模型不断对多个栅格组合计算出的损失值进行优化，以寻求最优解。该最优解对应的多个栅格在实际环境中对应的空间位置所连成的路径即为自动避障路径。
请继续参考图3，假设无人机从障碍物的上方绕过障碍物需要向上攀升的距离为d1，无人机从右侧绕过障碍物需要飞行的距离为d2，无人机从左侧绕过障碍物需要飞行的距离为d3，且d1小于d2小于d3。如果追求最短路径，无人机会选择向上攀升来避障，但是由于避障代价地图中左右偏移一个栅格的避障代价最小，向上偏移一个栅格的避障代价其次，向下偏移一个栅格的避障代价无穷大。如此，即使向上攀升飞行距离最短也不一定会将其作为自动避障路径，而是还会综合考虑避障消耗的电量和通过安全性。
可选地，在得到自动避障路径之后，为了让用户清楚地了解到无人机将要如何绕开障碍物，可以将自动避障路径投影到实时环境图像，在遥控设备中显示投影后的实时环境图像。如图4所示，是一种通过投影的方式在实时环境图像中显示自动避障路径的示意图。图中的五角星用于表示无人机将要先后到达的空间位置，多个五角星连成的线表示投影到实时环境图像中的自动避障路径，箭头方向用于指示无人机的飞行方向。
需要说明的是，向实时环境图像投影自动避障路径的过程与向实时环境图像投影避障代价地图的过程类似，在此不再详细阐述投影自动避障路径的过程，具体投影自动避障路径的过程可以参照前面描述投影避障代价地图的过程。
另外值得注意的是，将自动避障路径投影到实时环境图像的过程可以由无人机实现，或者也可以由遥控设备实现。如果是由遥控设备来实现，则无人机可以将投影所需的投影参数发送给遥控设备，由遥控设备基于投影所需的投影参数将自动避障路径投影到实时环境图像中。其中，投影所需的投影参数包括前述坐标转换所需的参数，如拍摄实时环境图像时无人机的位姿、图像拍摄装置的位姿，图像拍摄装置相对于无人机的位姿，云台相对于无人机的位姿、以及图像拍摄装置相对于云台的位姿中的一个或多个。
可以理解的是，在本发明实施例提供的方法中，无需在无人机飞行的全过 程中在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记。在某些情况下，无需对第一图像区域以及第二图像区域进行标记，否则会在实时环境图像中增加不必要的干扰。
基于此，可选地，可以在无人机和障碍物的距离满足第一预设条件时，执行在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记的操作。
其中，上述第一预设条件例如可以是第一距离范围区间。在无人机和障碍物的距离位于第一距离范围区间时，可以在实时环境图像的第一图像区域叠加第一标记，以及在实时环境图像的第二图像区域叠加第二标记的操作。
另外，可选地，向实时环境图像投影自动避障路径的过程也可以在无人机和障碍物的距离满足第一预设条件时触发执行。如图5所示，在无人机和障碍物的距离满足第一预设条件时，可以在实时环境图像中叠加展示投影后的自动避障路径，以供用户查看。
进一步地，在无人机和障碍物的距离满足第一预设条件时，还可以输出提示信息以提示用户无人机正在按照自动避障路径执行自动避障操作。以此来让用户感知无人机当前的工作状态为正在执行自动避障操作，且无人机将要按照在实时环境图像中叠加展示投影后的自动避障路径那样进行自动避障操作。
可选地，在无人机和障碍物的距离满足第二预设条件时，输出提示信息以提示用户无人机距离障碍物过近，无人机将执行减速和/或爬升操作。
其中，上述第二预设条件例如可以是第二距离范围区间，第二距离范围区间的最大值小于前述第一距离范围区间的最小值。在无人机和障碍物的距离位于第二距离范围区间时，可以输出提示信息以提示用户无人机距离障碍物过近，无人机将执行减速和/或爬升操作。
需要说明的是，当无人机与障碍物的距离过近时，由于无人机的机械性能等因素的限制，无人机难以以平滑的路径自动绕开障碍物。此时，无人机可以直接选择通过减速和爬升的操作进行避障。
如图6所示，当无人机和障碍物的距离满足第二预设条件时，可以在原有 的实时环境图像的基础上，在实时环境图像的右上角部分相应添加一个图中所示的图标，该图标是可以是一个告警标识，用于提示用户无人机距离障碍物过近。
可选地，在无人机和障碍物的距离满足第三预设条件时，输出提示信息以提示用户检测到远距离障碍物，安全可直行。
其中，上述第三预设条件例如可以是第三距离范围区间，第三距离范围区间的最小值大于前述第一距离范围区间的最大值。在无人机和障碍物的距离位于第三距离范围区间时，表示无人机距离障碍物比较远，相应可以输出提示信息以提示用户检测到远距离障碍物，安全可直行。
如图7所示，当无人机和障碍物的距离满足第三预设条件时，可以在原有的实时环境图像的基础上，在实时环境图像的右上角部分相应添加一个图中所示的图标，该图标例如是一个笑脸标识，用于提示用户当前安全可直行。
采用本发明，可以在实时环境图像中的不同图像区域叠加不同的标记，不同图像区域对应的环境空间具有的避障代价不同。在无人机自动避障的应用场景中，通过对不同图像区域叠加的不同标记能够让用户更加直观地了解无人机的避障逻辑。在用户手动操作无人机避障的应用场景中，对不同图像区域叠加的不同标记有利于辅助用户进行更加安全高效的避障控制操作。
本发明又一示例性实施例提供了一种无人机避障装置，如图8所示，该装置包括：
存储器1910，用于存储计算机程序；
处理器1920，用于运行存储器1910中存储的计算机程序以实现：
获取无人机的图像拍摄装置拍摄的实时环境图像；
在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述第一图像区域不同于所述第二图像区域，所述第一标记不同于所述第二标记；
其中，所述第一标记和所述第二标记用于指示所述第一图像区域对应的环 境空间和所述第二图像区域对应的环境空间具有不同的避障代价，所述避障代价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。
可选地，所述通过安全性越差，所述避障代价越大；
所述避障消耗的电量越大，所述避障代价越大；和/或
所述避障路径的长度越长，所述避障代价越大。
可选地，所述第一图像区域位于障碍物图像区域的上方，所述第二图像区域位于所述障碍物图像区域的两侧。
可选地，所述第一图像区域位于障碍物图像区域的上方，所述第二图像区域位于所述障碍物图像区域的下方。
可选地，所述标记为颜色标记、文字标记、纹理标记中的一个或多个。
可选地，所述在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，包括：
根据避障代价地图在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述避障代价地图是基于所述无人机的探测装置探测的数据得到的。
可选地，所述无人机的自动避障路径是基于所述避障代价地图规划得到的。
可选地，所述处理器1920，用于：
在所述无人机和障碍物的距离满足第一预设条件时，执行在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记的操作。
可选地，所述处理器1920，还用于：
在所述无人机和障碍物的距离满足所述第一预设条件时，将所述无人机的自动避障路径投影到所述实时环境图像中。
可选地，所述处理器1920，还用于：
在所述无人机和障碍物的距离满足所述第一预设条件时，输出提示信息以提示用户所述无人机正在按照所述自动避障路径执行自动避障操作。
可选地，所述处理器1920，还用于：
在所述无人机和障碍物的距离满足第二预设条件时，输出提示信息以提示用户所述无人机距离障碍物过近，所述无人机将执行减速和/或爬升操作。
可选地，所述处理器1920，还用于：
在所述无人机和障碍物的距离满足第三预设条件时，输出提示信息以提示用户检测到远距离障碍物，安全可直行。
图8所示的无人机避障装置可以执行图1-图7所示实施例的方法，本实施例未详细描述的部分，可参考对图1-图7所示实施例的相关说明。该技术方案的执行过程和技术效果参见图1-图7所示实施例中的描述，在此不再赘述。
本发明实施例还提供一种无人机，该无人机可以包括图8所示实施例中提供的无人机避障装置。
本发明再一实施例还提供一种无人机的遥控设备，该遥控设备可以包括图8所示实施例中提供的无人机避障装置。
另外，本发明实施例还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有可执行代码，所述可执行代码用于实现如前述各实施例提供的无人机避障方法。
以上各个实施例中的技术方案、技术特征在不相冲突的情况下均可以单独，或者进行组合，只要未超出本领域技术人员的认知范围，均属于本发明保护范围内的等同实施例。
以上所述仅为本发明的实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。
Claims (27)
- 一种无人机避障方法，其特征在于，所述方法包括：获取无人机的图像拍摄装置拍摄的实时环境图像；在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述第一图像区域不同于所述第二图像区域，所述第一标记不同于所述第二标记；其中，所述第一标记和所述第二标记用于指示所述第一图像区域对应的环境空间和所述第二图像区域对应的环境空间具有不同的避障代价，所述避障代价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。
- 根据权利要求1所述的方法，其特征在于：所述通过安全性越差，所述避障代价越大；所述避障消耗的电量越大，所述避障代价越大；和/或所述避障路径的长度越长，所述避障代价越大。
- 根据权利要求1所述的方法，其特征在于，所述第一图像区域位于障碍物图像区域的上方，所述第二图像区域位于所述障碍物图像区域的两侧。
- 根据权利要求1所述的方法，其特征在于，所述第一图像区域位于障碍物图像区域的上方，所述第二图像区域位于所述障碍物图像区域的下方。
- 根据权利要求1所述的方法，其特征在于，所述标记为颜色标记、文字标记、纹理标记中的一个或多个。
- 根据权利要求1所述的方法，其特征在于，所述在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，包括：根据避障代价地图在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述避障代价地图是基于所述无人机的探测装置探测的数据得到的。
- 根据权利要求6所述的方法，其特征在于，所述无人机的自动避障路径 是基于所述避障代价地图规划得到的。
- 根据权利要求1-7任一项所述的方法，其特征在于，所述方法还包括：在所述无人机和障碍物的距离满足第一预设条件时，执行在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记的操作。
- 根据权利要求8所述的方法，其特征在于，所述方法还包括：在所述无人机和障碍物的距离满足所述第一预设条件时，将所述无人机的自动避障路径投影到所述实时环境图像中。
- 根据权利要求9所述的方法，其特征在于，所述方法还包括：在所述无人机和障碍物的距离满足所述第一预设条件时，输出提示信息以提示用户所述无人机正在按照所述自动避障路径执行自动避障操作。
- 根据权利要求8所述的方法，其特征在于，所述方法还包括：在所述无人机和障碍物的距离满足第二预设条件时，输出提示信息以提示用户所述无人机距离障碍物过近，所述无人机将执行减速和/或爬升操作。
- 根据权利要求8所述的方法，其特征在于，所述方法还包括：在所述无人机和障碍物的距离满足第三预设条件时，输出提示信息以提示用户检测到远距离障碍物，安全可直行。
- 一种无人机避障装置，其特征在于，包括存储器、处理器；其中，所述存储器上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器实现：获取无人机的图像拍摄装置拍摄的实时环境图像；在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述第一图像区域不同于所述第二图像区域，所述第一标记不同于所述第二标记；其中，所述第一标记和所述第二标记用于指示所述第一图像区域对应的环境空间和所述第二图像区域对应的环境空间具有不同的避障代价，所述避障代 价与通过安全性、避障消耗的电量和/或避障路径的长度相关联。
- 根据权利要求13所述的装置，其特征在于：所述通过安全性越差，所述避障代价越大；所述避障消耗的电量越大，所述避障代价越大；和/或所述避障路径的长度越长，所述避障代价越大。
- 根据权利要求13所述的装置，其特征在于，所述第一图像区域位于障碍物图像区域的上方，所述第二图像区域位于所述障碍物图像区域的两侧。
- 根据权利要求13所述的装置，其特征在于，所述第一图像区域位于障碍物图像区域的上方，所述第二图像区域位于所述障碍物图像区域的下方。
- 根据权利要求13所述的装置，其特征在于，所述标记为颜色标记、文字标记、纹理标记中的一个或多个。
- 根据权利要求13所述的装置，其特征在于，所述在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，包括：根据避障代价地图在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记，所述避障代价地图是基于所述无人机的探测装置探测的数据得到的。
- 根据权利要求18所述的装置，其特征在于，所述无人机的自动避障路径是基于所述避障代价地图规划得到的。
- 根据权利要求13-19任一项所述的装置，其特征在于，所述处理器，用于：在所述无人机和障碍物的距离满足第一预设条件时，执行在所述实时环境图像的第一图像区域叠加第一标记，以及在所述实时环境图像的第二图像区域叠加第二标记的操作。
- 根据权利要求20所述的装置，其特征在于，所述处理器，还用于：在所述无人机和障碍物的距离满足所述第一预设条件时，将所述无人机的自动避障路径投影到所述实时环境图像中。
- 根据权利要求21所述的装置，其特征在于，所述处理器，还用于：在所述无人机和障碍物的距离满足所述第一预设条件时，输出提示信息以提示用户所述无人机正在按照所述自动避障路径执行自动避障操作。
- 根据权利要求20所述的装置，其特征在于，所述处理器，还用于：在所述无人机和障碍物的距离满足第二预设条件时，输出提示信息以提示用户所述无人机距离障碍物过近，所述无人机将执行减速和/或爬升操作。
- 根据权利要求20所述的装置，其特征在于，所述处理器，还用于：在所述无人机和障碍物的距离满足第三预设条件时，输出提示信息以提示用户检测到远距离障碍物，安全可直行。
- 一种无人机，其特征在于，包括如13-24中任一项所述无人机避障装置。
- 一种无人机的遥控设备，其特征在于，包括如13-24中任一项所述无人机避障装置。
- 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有程序指令，所述程序指令用于实现权利要求1-12中任一项所述的无人机避障方法。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2022/084823 WO2023184487A1 (zh) | 2022-04-01 | 2022-04-01 | 无人机避障方法、装置、无人机、遥控设备和存储介质 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2022/084823 WO2023184487A1 (zh) | 2022-04-01 | 2022-04-01 | 无人机避障方法、装置、无人机、遥控设备和存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023184487A1 true WO2023184487A1 (zh) | 2023-10-05 |
Family
ID=88198785
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2022/084823 WO2023184487A1 (zh) | 2022-04-01 | 2022-04-01 | 无人机避障方法、装置、无人机、遥控设备和存储介质 |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2023184487A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117930870A (zh) * | 2024-03-21 | 2024-04-26 | 天津万森科技发展有限公司 | 一种无人机的实时避障方法及系统 |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105517666A (zh) * | 2014-09-05 | 2016-04-20 | 深圳市大疆创新科技有限公司 | 基于情景的飞行模式选择 |
| CN108521808A (zh) * | 2017-10-31 | 2018-09-11 | 深圳市大疆创新科技有限公司 | 一种障碍信息显示方法、显示装置、无人机及系统 |
| CN108521807A (zh) * | 2017-04-27 | 2018-09-11 | 深圳市大疆创新科技有限公司 | 无人机的控制方法、设备及障碍物的提示方法、设备 |
| CN109074096A (zh) * | 2017-12-29 | 2018-12-21 | 深圳市大疆创新科技有限公司 | 一种无人机、控制终端的控制方法及相关设备 |
| CN109358638A (zh) * | 2018-09-10 | 2019-02-19 | 南京航空航天大学 | 基于分布式地图的无人机视觉避障方法 |
| CN110244760A (zh) * | 2019-06-06 | 2019-09-17 | 深圳市道通智能航空技术有限公司 | 一种避障方法、装置和电子设备 |
| WO2020000479A1 (zh) * | 2018-06-30 | 2020-01-02 | 深圳市大疆创新科技有限公司 | 一种信息提示方法及控制终端 |
| CN112771464A (zh) * | 2020-04-27 | 2021-05-07 | 深圳市大疆创新科技有限公司 | 飞行提示方法、装置、控制终端、系统及存储介质 |
| CN113795803A (zh) * | 2020-08-17 | 2021-12-14 | 深圳市大疆创新科技有限公司 | 无人飞行器的飞行辅助方法、设备、芯片、系统及介质 |
-
2022
- 2022-04-01 WO PCT/CN2022/084823 patent/WO2023184487A1/zh unknown
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105517666A (zh) * | 2014-09-05 | 2016-04-20 | 深圳市大疆创新科技有限公司 | 基于情景的飞行模式选择 |
| CN108521807A (zh) * | 2017-04-27 | 2018-09-11 | 深圳市大疆创新科技有限公司 | 无人机的控制方法、设备及障碍物的提示方法、设备 |
| CN108521808A (zh) * | 2017-10-31 | 2018-09-11 | 深圳市大疆创新科技有限公司 | 一种障碍信息显示方法、显示装置、无人机及系统 |
| CN109074096A (zh) * | 2017-12-29 | 2018-12-21 | 深圳市大疆创新科技有限公司 | 一种无人机、控制终端的控制方法及相关设备 |
| WO2020000479A1 (zh) * | 2018-06-30 | 2020-01-02 | 深圳市大疆创新科技有限公司 | 一种信息提示方法及控制终端 |
| CN109358638A (zh) * | 2018-09-10 | 2019-02-19 | 南京航空航天大学 | 基于分布式地图的无人机视觉避障方法 |
| CN110244760A (zh) * | 2019-06-06 | 2019-09-17 | 深圳市道通智能航空技术有限公司 | 一种避障方法、装置和电子设备 |
| CN112771464A (zh) * | 2020-04-27 | 2021-05-07 | 深圳市大疆创新科技有限公司 | 飞行提示方法、装置、控制终端、系统及存储介质 |
| CN113795803A (zh) * | 2020-08-17 | 2021-12-14 | 深圳市大疆创新科技有限公司 | 无人飞行器的飞行辅助方法、设备、芯片、系统及介质 |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117930870A (zh) * | 2024-03-21 | 2024-04-26 | 天津万森科技发展有限公司 | 一种无人机的实时避障方法及系统 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112650255B (zh) | 基于视觉与激光雷达信息融合的机器人定位导航方法 | |
| JP6601554B2 (ja) | 無人航空機、無人航空機制御システム、飛行制御方法およびコンピュータプログラム | |
| US20170259920A1 (en) | Automatic painting system with drone, user interface and computer vision | |
| US8725273B2 (en) | Situational awareness for teleoperation of a remote vehicle | |
| WO2016033797A1 (en) | Multi-sensor environmental mapping | |
| CN111338382B (zh) | 一种安全态势引导的无人机路径规划方法 | |
| US20200012756A1 (en) | Vision simulation system for simulating operations of a movable platform | |
| CN112114594B (zh) | 一种基于视觉与性能约束的多无人机协同控制方法及系统 | |
| JP2018156491A (ja) | 設備点検システム | |
| US10739792B2 (en) | Trajectory control of a vehicle | |
| WO2018179404A1 (ja) | 情報処理装置、情報処理方法、および情報処理プログラム | |
| WO2022095060A1 (zh) | 路径规划方法、路径规划装置、路径规划系统和介质 | |
| US10565887B2 (en) | Flight initiation proximity warning system | |
| CN106303417B (zh) | 用于无人平台的增强全景监控方法 | |
| WO2023077341A1 (zh) | 无人机的返航方法、装置、无人机、遥控设备、系统及存储介质 | |
| WO2023184487A1 (zh) | 无人机避障方法、装置、无人机、遥控设备和存储介质 | |
| CN112596071A (zh) | 无人机自主定位方法、装置及无人机 | |
| CN111443723A (zh) | 一种无人机第三视角视景生成和显示程序 | |
| CN112379681A (zh) | 无人机避障飞行方法、装置及无人机 | |
| WO2022095067A1 (zh) | 路径规划方法、路径规划装置、路径规划系统和介质 | |
| CN112378397A (zh) | 无人机跟踪目标的方法、装置及无人机 | |
| JP2021162572A (ja) | 位置算出方法及び情報処理システム | |
| WO2021237462A1 (zh) | 无人飞行器的限高方法、装置、无人飞行器及存储介质 | |
| WO2023102911A1 (zh) | 数据采集方法、数据展示方法、数据处理方法、飞行器的降落方法、数据展示系统及存储介质 | |
| CN112964263B (zh) | 自动建图方法、装置、移动机器人及可读存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22934323 Country of ref document: EP Kind code of ref document: A1 |