WO2021168678A1

WO2021168678A1 - Path planning method, tracking device, storage medium, and computer program product

Info

Publication number: WO2021168678A1
Application number: PCT/CN2020/076729
Authority: WO
Inventors: 郭晓东; 吴博
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-02-26
Filing date: 2020-02-26
Publication date: 2021-09-02
Also published as: CN112513766A

Abstract

Disclosed are a path planning method, a tracking device, a storage medium, and a computer program product. The method comprises: acquiring movement information of a target and the position of an obstacle in an environment where the target is located, wherein the movement information is acquired on the basis of a vision sensor of a tracking device (S110); and planning a tracking path of the tracking device for the target according to the movement information of the target and the position of the obstacle, such that the target keeps within an observation range of the vision sensor when the tracking device moves along the tracking path (S120). The tracking path planned by means of the method enables an obstacle to be far away from the observation sight of a tracking device for a tracked target, thereby helping to improve the tracking success rate of a tracking device for a target.

Description

Path planning method, tracking device, storage medium and computer program product

Copyright statement

The content disclosed in this patent document contains copyrighted material. The copyright belongs to the copyright owner. The copyright owner does not object to anyone copying the patent document or the patent disclosure in the official records and archives of the Patent and Trademark Office.

Technical field

This application relates to the field of path planning, and more specifically, to a method, tracking device, storage medium, and computer program product for path planning.

Background technique

At present, the application of intelligent tracking technology is becoming more and more extensive. For example, drones can be used to intelligently follow targets such as people, cars, and boats to monitor the status of the target; another example, underwater robots can be used to track and camera fish to learn about the habits of the fish.

Traditional intelligent tracking technology has the problem of low tracking success rate. Therefore, how to improve the tracking success rate of the tracking device to the target has always been a research hotspot in this field.

Summary of the invention

The present application provides a path planning method, tracking device, computer readable storage medium, and computer program product, which can improve the success rate of target tracking by the tracking device.

In a first aspect, a method for path planning is provided, which includes: obtaining movement information of a target and the position of an obstacle in an environment in which the target is located, wherein the movement information is obtained based on a visual sensor of a tracking device; The motion information of the target and the position of the obstacle are planned, and the tracking path of the tracking device to the target is planned, so that when the tracking device moves along the tracking path, the target is kept in the visual field. Within the observation range of the sensor.

In a second aspect, a tracking device is provided, including: a memory for storing a program; and a processor for executing the method as described in the first aspect.

In a third aspect, a tracking device is provided, which includes a module for executing the method described in the first aspect.

In a fourth aspect, a machine-readable storage medium is provided, on which instructions for executing the method described in the first aspect are stored.

In a fifth aspect, a computer program product is provided, including instructions for executing the method as described in the first aspect.

The present application can keep the target within the observation range of the visual sensor when the tracking device moves along the tracking path, thereby helping to improve the tracking success rate of the tracking device.

Description of the drawings

FIG. 1 is a schematic flowchart of a path planning method provided by an embodiment of the present application.

FIG. 2 is a flowchart of a possible implementation manner of step S120 in FIG. 1.

FIG. 3 is a schematic flowchart of a specific implementation manner of the embodiment in FIG. 2.

FIG. 4 is a schematic flowchart of another specific implementation manner of the embodiment in FIG. 2.

Fig. 5 is a structural diagram of a tracking device provided by an embodiment of the present application.

Fig. 6 is a structural diagram of a tracking device provided by another embodiment of the present application.

Detailed ways

In the related technology, in the process of tracking a target using a tracking device, the problem of tracking failure (that is, the tracking target is lost) is prone to occur. After careful study, it is found that the problem mainly exists: when the related technology is planning the tracking path, the main concern is whether the tracking device may collide with the obstacle when the tracking device moves along the tracking path, but ignores the tracking device’s impact on the target. The observation line of sight may be blocked by obstacles. In fact, if the tracking path planning results in the tracking device's observing sight of the target being blocked by obstacles, it may cause the tracking device to fail to observe or recognize the target, resulting in tracking failure.

In order to facilitate understanding, a possible implementation of the tracking function in related technologies is given below by taking a drone as an example.

The tracking function can be divided into three functional modules: machine learning module, target estimation module, and tracking planning module.

The machine learning module can use the images collected by the camera to identify the target. The target estimation module can use the output result of the machine learning module and the current observation information of the drone to estimate the movement information of the target (such as the position and/or speed of the target) online.

The tracking planning module can use the estimated position and/or speed of the target to plan the tracking path of the tracking device.

After obtaining the tracking path, the UAV can use the flight control system to control the UAV to move along the tracking path to keep the UAV tracking the target.

In the tracking process, the above implementation may use the UAV's obstacle avoidance function to avoid obstacles that may collide with the UAV, but the above implementation does not consider whether the UAV's observation line of sight to the target will be obstructed. Object occlusion, leading to the realization of this tracking technology is very easy to lose the tracking target in a scene with many obstacles.

The embodiment of the present application provides a path planning method to improve the tracking success rate of the tracking device. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a path planning method provided by an embodiment of the present application. The method in Figure 1 can be executed by a tracking device.

The tracking device may be any device capable of tracking a target through machine vision. In some embodiments, the tracking device may be a movable platform. The movable platform may be, for example, one of the following equipment: drones, unmanned vehicles, underwater robots or walking robots. The method in FIG. 1 may include step S110 and step S120. These steps are described in detail below.

In step S110, the motion information of the target and the position of the obstacle in the environment where the target is located are acquired, wherein the motion information is acquired based on the visual sensor of the tracking device.

The vision sensor can be a monocular/binocular or a multi-eye vision sensor. The visual perception spectrum is not limited to the visible spectrum, it can also be an infrared spectrum, or even a combination of multiple spectra.

In step S120, plan the tracking path of the tracking device to the target according to the movement information of the target and the position of the obstacle, so that when the tracking device moves along the tracking path, the target is kept within the observation range of the vision sensor .

The present application can keep the target within the observation range of the vision sensor when the tracking device moves along the tracking path, thereby facilitating the vision sensor to obtain the movement information of the target and further improving the tracking success of the tracking device Rate.

Specifically, the tracking path planned in the embodiment of the present application keeps the obstacle away from the observation line of sight of the tracked target by the tracking device.

The motion information of the target may include one or more of the following information: historical motion information of the target, current motion information of the target, and future motion information of the target. The future motion information of the target can be obtained through prediction. For example, it is possible to estimate the movement trend of the target according to the historical movement information of the target and/or the current movement information, so as to determine the position and/or speed of the target at a certain time in the future. The embodiments of this application do not specifically limit the content of the target's motion information. In some embodiments, the target's motion information may include one or more of the following: position information, speed information, acceleration information, and orientation information , Posture information. In some embodiments, the tracking device may be provided with a target estimation module as shown in FIG. 5, and the motion information of the target may be completed by the target estimation module.

There are many ways to define obstacles in the environment where the target is located. In some embodiments, the obstacles in the environment where the target is located may include obstacles around the tracking device and/or obstacles around the target.

Whether it is the specific form of motion information or the selection range of obstacles, it can be flexibly set according to the specific conditions of the tracking task. For example, when accurate tracking is required, not only the position and speed of the target must be considered, but also the acceleration and attitude of the target; when performing ordinary tracking, only the position and speed of the target can be considered, without considering the acceleration and attitude of the target . For another example, when the target is in a complex environment with many obstacles, the tracking equipment and/or obstacles within the preset range of the target's forward direction can be considered to plan the tracking path in advance; when the target is simple, In an environment with fewer obstacles, only the obstacles in the forward direction that are closest to the tracking device and/or the target can be considered, so as to plan the current tracking path.

There are many ways to obtain the location information of the obstacle. In some embodiments, the location of the obstacle can be found in the map data. For example, the tracking device uses machine vision to sense the environment, continuously reconstructs a 3D map of the surrounding environment, and uses the 3D map to locate obstacles. In other embodiments, the location of the obstacle may also be acquired based on the sensor of the tracking device. For example, the radar or depth camera on the tracking device can be used to detect surrounding objects to obtain the location of obstacles.

The planning module relies on the map module to judge the obstacles around the drone. UAVs use depth sensors (such as binocular systems) to build the surrounding environment. Through the local map, you can query whether a coordinate point is occupied by an obstacle, or you can query a coordinate point and the distance to the nearest obstacle.

In some embodiments, when planning the tracking path, not only the motion information of the target may be considered, but also the motion information of the tracking device may be considered. In this embodiment, the method of FIG. 1 may further include: acquiring movement information of the tracking device. Step S120 in FIG. 1 can be replaced with: planning the tracking path of the tracking device to the target according to the movement information of the target, the location of the obstacle, and the movement information of the tracking device.

The motion information of the tracking device may include one or more of the following information of the tracking device: position information, speed information, acceleration information, orientation information, and posture information.

The embodiment of the present application further considers the movement information of the tracking device, which can effectively avoid the problem that the planned tracking path does not match the current movement state of the tracking device, and makes the planned tracking path more reasonable. For example, when planning a tracking path, consider the current position and speed of the tracking device, so that the tracking device can smoothly transition from the current motion state to a motion state that can track the target along the tracking path, avoiding trajectory conflicts and position during the transition process / Speed change and other issues.

Step S120 can be implemented in multiple ways. For example, a tracking path that can avoid obstacles can be directly planned, or initial path planning information can be generated first, and then the final tracking path setting can be calculated according to the initial path planning information. In the following, an example of the implementation of step S120 will be described with reference to FIG. 2.

FIG. 2 is a schematic flowchart of a possible implementation manner of step S120. In FIG. 2, step S120 may include step S122 to step S126. These steps are described in detail below.

In step S122, initial path planning information is generated at least according to the movement information of the target.

In some embodiments, the initial path planning information may also be generated based on the movement information of the target and the movement information of the tracking device.

In step S124, the cost of obstructing the observation line of sight of the tracking device to the target by the obstacle is determined according to the position of the obstacle and the initial path planning information.

Wherein, the observation line of sight may refer to a narrower range of the observation range of the visual sensor of the tracking device. The light emitted/reflected by the target is imaged on the sensing element of the vision sensor along the observation line of sight. Further, the tracking device can calculate the movement information of the target according to the imaging change of the target.

Optionally, in some embodiments, the cost is determined based on distance information between the obstacle and the observation line of sight.

The cost of the observing line of sight of the tracking device to the target being obscured by obstacles can be used to characterize whether the observing line of sight is obscured by obstacles, or to characterize the degree to which the observing line of sight is blocked by obstacles, or to characterize that the observing line of sight is blocked by obstacles Possibility of occlusion. For example, in some embodiments, the higher the cost, the greater the degree (or probability) that the observation line of sight is blocked by obstacles.

In step S126, a tracking path is planned based on the cost of observing the line of sight being blocked by obstacles and the initial path planning information.

In the process of planning the tracking path, the embodiment of the present application considers the cost of the observing sight of the tracking device to the target being blocked by obstacles, thereby helping to obtain a tracking path with less or no obstruction by obstacles, thereby improving the performance of the tracking device. Track success rate.

In step S126, in addition to the cost related to obstruction by obstacles, other costs may also be considered when planning the tracking path. Such as obstacle avoidance cost and/or distance cost. Among them, the obstacle avoidance cost can be used to indicate the possibility of a collision between the tracking device and an obstacle; the distance cost can be used to indicate the difference between the tracking parameters of the tracking device and the desired tracking parameters.

The aforementioned tracking parameters may include one or more of the following parameters: tracking distance, tracking height, and tracking angle. The desired tracking parameters can be configured according to actual tracking tasks or tracking requirements. The desired tracking parameters can be preset values or input by the user of the tracking device.

There may be multiple ways to define the initial path planning information, which is not limited in the embodiment of the present application. For example, the initial path planning information may include information on multiple initial paths; and/or the initial path information may include information on multiple initial target points (the end points of the preliminary tracking path). The definition of the initial path planning information is different, and the specific implementation of Fig. 2 is also different. Two examples are given below in conjunction with Figs. 3 and 4.

In the embodiment corresponding to FIG. 3, the initial path planning information includes multiple initial path information. Step S322, step S324, and step S326 in FIG. 3 correspond to step S122, step 124, and step S126 in FIG. 2, respectively. These steps are described in detail below.

In step S322, multiple initial paths are generated at least according to the movement information of the target.

For example, the tracking device can directly generate multiple initial paths based on the movement information of the target. For another example, the tracking device may comprehensively consider its own motion information, target motion information, and/or desired tracking parameters, etc., before generating multiple initial paths.

In addition, in some embodiments, a reference path generation module as shown in FIG. 5 may be provided for the tracking device, and the function of generating multiple initial paths may be completed by the reference path generation module.

In step S324, according to the position of the obstacle, the cost of the observation line of sight corresponding to each of the multiple initial paths being blocked by the obstacle is determined.

In other words, each initial path can correspond to a cost. Since the relative positions of different initial paths and obstacles are not exactly the same, the situation of the target being blocked by obstacles may be different when the tracking device moves along different initial paths, which may result in different The initial path corresponds to different costs.

There are many ways to calculate the cost corresponding to each initial path. For example, the cost corresponding to each initial path can be calculated according to the minimum distance between the initial path and the obstacle, the smaller the distance, the higher the cost; or, the corresponding cost of each initial path can be calculated according to the number of obstacles in a certain range around the initial path The higher the number of obstacles, the higher the cost; or, the cost corresponding to each initial path can be calculated by comprehensively considering the above factors, and different factors can be set with the same or different weights.

In step S326, a tracking path is selected from the multiple initial paths based on the cost of the observation line of sight corresponding to each of the multiple initial paths being blocked by obstacles.

In some embodiments, the initial path with the smallest corresponding cost among the plurality of initial paths may be determined as the tracking path. In other embodiments, any one of the multiple initial paths whose corresponding cost is less than the preset threshold may be determined as the tracking path.

In the embodiment corresponding to FIG. 4, the initial path planning information includes information of multiple initial target points. Step S422, step S424, and step S426 in FIG. 4 correspond to step S122, step 124, and step S126 in FIG. 2, respectively. These steps are described in detail below.

In step S422, a plurality of initial target points are determined at least according to the motion information of the target.

For example, the tracking device can directly generate multiple initial target points based on the motion information of the target. For another example, the tracking device may comprehensively consider its own motion information, target motion information, and/or desired tracking parameters, etc., before generating multiple initial target points.

In addition, in some embodiments, a reference path generation module as shown in FIG. 5 may be provided for the tracking device, and the function of generating multiple initial target points may be completed by the reference path generation module.

In step S424, according to the position of the obstacle, the cost of the observation line of sight corresponding to each of the multiple initial target points being blocked by the obstacle is determined.

In other words, each initial target point can correspond to a cost. Because the positions of different initial target points are different, the situation of the target being blocked by obstacles may be different when the tracking device moves to different initial target points, which may result in different initial target points. The target points correspond to different costs.

There are many ways to calculate the cost corresponding to each initial target point. For example, the distance between each point on the line between the starting point and the initial target point of the tracking device and the obstacle can be calculated according to the position of the obstacle. Integrate to obtain the cost of obstructing the observation line of sight corresponding to multiple initial target points. Alternatively, the distance between the line between the initial target point and the target and the obstacle can be determined; according to the distance between the line and the obstacle, and the mapping relationship between the preset distance and the cost, multiple initial target points can be obtained The corresponding observation line of sight is blocked by obstacles. Or, according to the position of the obstacle, determine each sampling point on the line between the starting point and the initial target point of the tracking device; when calculating that the tracking device is at the sampling point, the observation line of sight is subject to The cost of occlusion: integrating the costs corresponding to the multiple sampling points to obtain the cost at which the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle.

In step S426, a tracking path is planned based on the cost of the observation line of sight corresponding to each of the multiple initial target points being blocked by obstacles.

For example, the target point with the lowest cost can be selected from multiple candidate target points; the tracking path can be planned according to the target point with the lowest cost. For another example, any target point whose corresponding cost meets a preset threshold can be selected from a plurality of candidate target points, and a tracking path can be planned according to the target point.

The cost of the initial target point may not only include the cost of obstacle occlusion described above (hereinafter referred to as occlusion cost), but also include obstacle avoidance cost and/or distance cost. A possible calculation method for the cost of the initial target point is given below.

The cost J _{total of the} initial target point satisfies formula (1):

J _total ＝J _Obs +J _dis +J _occu (1)

In formula (1), J _Obs represents the cost of avoiding obstacles, J _dis represents the cost of distance, and _Joccu represents the cost of occlusion.

The obstacle avoidance cost J _Obs satisfies formula (2):

J _Obs =λ _Obs C _Obs (2)

_{Λ Obs} in the formula (2) represents the weight of the obstacle avoidance cost in the cost of the initial target point. The value of the weight can be input by the user, or the default value can be adopted.

_{C Obs} in formula (2) satisfies formula (3):

In formula (3), P _u represents the location of the tracking device, and _Me (P _u ) is the distance between the closest obstacle to the tracking device and the tracking device. ε ₀ represents the cutoff distance. The value of the cutoff distance is related to the shape and size of the tracking device. It can be set by the user or the default value can be used. Taking the tracking device as an unmanned aerial vehicle as an example, _{the value of ε 0} can be 0.5 meters.

Combining formulas (2) and (3), it can be seen that the setting of obstacle avoidance cost helps to select the initial target point that will not collide with obstacles.

In this example, the distance cost J _dis satisfies formula (4):

J _dis ＝λ _dis C _dis (4)

_{The λ dis} in formula (4) represents the weight of the distance cost in the cost of the initial target point. The value of the weight can be input by the user, or the default value can be used. C _dis represents the distance between the initial target point and the desired target point.

It can be seen from formula (4) that the setting of the distance cost helps to select the initial target point that is closest to the desired target point, thereby helping to better complete the tracking task.

In this example, the occlusion cost J _occu satisfies formula (5):

_{The λ occu} in formula (5) represents the weight of the occlusion cost in the cost of the initial target point. The value of the weight can be input by the user or a default value.

_{C occu} in formula (5) satisfies formula (6):

In the above formula, l _u represents the starting point of the tracking device (the current position of the tracking device can be used as the starting point, or a position near the current position of the tracking device can be used as the starting point) and the initial target point. Point on the path formed. When the value of τ is 0, l _u (τ) represents the starting point of the path; when the value of τ is 1, l _u (τ) represents the end point of the path (the end point is the initial target point). The value of τ from 0 to 1 means to traverse the points on the path. As for the occlusion cost, when the tracking device moves along the line between the starting point and the initial target point, the fewer obstacles between the tracking device and the target, the smaller the value of the occlusion cost. Therefore, the occlusion cost helps to select the initial target point with fewer obstacles between the tracking device and the target, thereby improving the tracking success rate.

There are many ways to calculate the cost of observing the line of sight blocked by obstacles, and formulas (5) and (6) are only an example. For example, in some embodiments, the occlusion cost may be expressed in a binary manner of 0/1. With a certain query radius, query the connection path from the initial target point to the target to be followed. The distance of the obstacle is less than the set value and it is considered to be occluded, and the distance greater than the set value is that it is not to be occluded. If occlusion occurs, a larger cost weight can be assigned to the initial target point where the occlusion occurs. For another example, in some embodiments, you can query the minimum distance between the initial target point and the tracked target and the obstacle distance, and then design an appropriate mapping function, so that the smaller the distance, the calculated initial target point The higher the cost, the more occluded initial target points are excluded as much as possible.

By designing a reasonable cost function of the occlusion item and selecting a reasonable candidate target point, avoiding the obstruction of the following target by obstacles on the flight path can make the UAV track the target stably for a long time, and it is not easy to follow it. In addition, the occlusion of the target by obstacles is reduced, the accuracy of visual estimation of the target position and speed is improved, and the stability and accuracy of following the target can be improved.

In some embodiments, the tracking device may be provided with a local obstacle avoidance module as shown in FIG. 5, and the path planning function described above may be completed by using the local obstacle avoidance module.

The foregoing describes in detail the method for tracking a target provided in the embodiment of the present application in conjunction with FIGS. 1 to 5, and the following describes in detail the tracking device provided in the embodiment of the present application in conjunction with FIG. 6. It should be understood that the description on the method side and the description on the device side correspond to each other, and repeated descriptions are appropriately omitted for brevity.

Fig. 6 is a schematic structural diagram of a tracking device provided by an embodiment of the present application. As shown in FIG. 6, the tracking device 600 includes a memory 610 and a processor 620.

The memory 610 can be used to store programs.

The processor 620 may be used to execute a program in the memory to perform the following operations: obtain the movement information of the target and the position of the obstacle in the environment where the target is located, wherein the movement information is obtained based on the visual sensor of the tracking device ; According to the movement information of the target and the position of the obstacle, plan the tracking path of the tracking device to the target, so that the tracking device keeps the target in place when the tracking device moves along the tracking path Within the observation range of the visual sensor.

Specifically, when the tracking device moves along the tracking path, the obstacle is kept away from the observation line of sight of the tracking device to the target.

Optionally, in some embodiments, the planning the tracking path of the tracking device to the target according to the movement information of the target and the position of the obstacle includes: generating an initial track based on at least the movement information of the target Path planning information; determine the cost of the observing line of sight of the tracking device to the target being blocked by the obstacle according to the location of the obstacle and the initial path planning information; based on the obstruction of the observing line of sight by the obstacle And the initial path planning information plan to obtain the tracking path.

Optionally, in some embodiments, the generating initial path planning information at least according to the movement information of the target includes: generating a plurality of initial paths at least according to the movement information of the target; The location and the initial path planning information determine the cost of obstructing the observation line of sight by the obstacle, including: determining that the observation line of sight corresponding to each of the plurality of initial paths is subject to the obstacle according to the position of the obstacle The cost of object occlusion; the planning to obtain the tracking path based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information includes: the observation line of sight corresponding to each of the multiple initial paths At the cost of being blocked by the obstacle, the tracking path is selected from the plurality of initial paths.

Optionally, in some embodiments, the selecting the tracking path from the plurality of initial paths based on the cost that the observation line of sight corresponding to each of the plurality of initial paths is blocked by the obstacle includes : Determine the initial path with the smallest corresponding cost among the multiple initial paths as the tracking path.

Optionally, in some embodiments, the generating initial path planning information at least according to the movement information of the target includes: determining a plurality of initial target points according to at least the movement information of the target; The location of the object and the initial path planning information determine the cost of obstructing the observation line of sight by the obstacle, including: determining that the observation line of sight corresponding to each of the multiple initial target points is subject to the obstruction according to the location of the obstacle. The cost of obstruction by the obstacle; the planning to obtain the tracking path based on the cost of the observation line of sight being obscured by the obstacle and the initial path planning information includes: based on the respective corresponding to the plurality of initial target points At the cost of obstructing the observation line of sight by the obstacle, the tracking path is obtained by planning.

Optionally, in some embodiments, the planning to obtain the tracking path based on the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle includes: The target point corresponding to the smallest cost is selected among the candidate target points; and the tracking path is planned according to the target point with the smallest cost.

Optionally, in some embodiments, the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle includes: according to the obstacle The position of the object, the distance between each point on the line between the starting point of the tracking device and the initial target point and the obstacle is integrated to obtain the observation line of sight corresponding to each of the multiple initial target points The cost of being blocked by the obstacle.

Optionally, in some embodiments, the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle includes: according to the obstacle The position of the object is determined, and each sampling point on the line between the starting point of the tracking device and the initial target point is determined; the cost of obstructing the observation line of sight when the tracking device is at the sampling point is calculated; The costs corresponding to the sampling points are integrated to obtain the costs at which the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle.

Optionally, in some embodiments, the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle includes: determining the initial The distance between the line between the target point and the target and the obstacle; according to the distance between the line and the obstacle, and the preset mapping relationship between the distance and the cost, the A cost at which the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle.

Optionally, in some embodiments, the tracking path planning is also based on obstacle avoidance cost and/or distance cost, and the obstacle avoidance cost is used to indicate the possibility of collision between the tracking device and an obstacle, and The distance cost is used to represent the difference between the tracking parameters of the tracking device and the desired tracking parameters.

Optionally, in some embodiments, the desired tracking parameter is a preset value or input by a user of the tracking device.

Optionally, in some embodiments, the tracking parameters include one or more of the following parameters: tracking distance, tracking height, and tracking angle.

Optionally, in some embodiments, the processor is further configured to perform the following operations: obtain movement information of the tracking device; and plan the tracking device according to the movement information of the target and the location of the obstacle The tracking path of the target includes: planning the tracking path of the tracking device to the target according to the motion information of the target, the location of the obstacle, and the motion information of the tracking device.

Optionally, in some embodiments, the motion information includes one or more of the following information: position information, speed information, acceleration information, orientation information, and posture information.

Optionally, in some embodiments, the obstacles include obstacles around the tracking device and/or obstacles around the target.

Optionally, in some embodiments, the acquiring the location of the obstacle in the environment where the target is located includes: finding the location of the obstacle in map data and/or acquiring based on the sensor of the tracking device The location of the obstacle.

Optionally, in some embodiments, the tracking device is a drone, an unmanned vehicle, an underwater robot, or a walking robot.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc. .

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for path planning, characterized in that it includes:

Acquiring movement information of the target and the location of obstacles in the environment where the target is located, wherein the movement information is acquired based on a visual sensor of a tracking device;

According to the movement information of the target and the position of the obstacle, the tracking path of the tracking device to the target is planned, so that when the tracking device moves along the tracking path, the target remains at the Within the observation range of the vision sensor.
The method according to claim 1, wherein the planning a tracking path for the target by a tracking device according to the motion information of the target and the position of the obstacle comprises:

Generating initial path planning information at least according to the movement information of the target;

Determining, according to the position of the obstacle and the initial path planning information, the cost of the observing line of sight of the tracking device to the target being blocked by the obstacle;

The tracking path is planned based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information.
The method according to claim 2, wherein the cost is determined based on distance information between the obstacle and the observation line of sight.
The method according to claim 2, wherein the generating initial path planning information at least according to the movement information of the target comprises:

Generate multiple initial paths at least according to the movement information of the target;

The determining the cost of obstructing the observation line of sight by the obstacle according to the position of the obstacle and the initial path planning information includes:

Determine the cost of obstructing the observation line of sight corresponding to each of the multiple initial paths by the obstacle according to the position of the obstacle;

The planning to obtain the tracking path based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information includes:

The tracking path is selected from the plurality of initial paths based on the cost that the observation line of sight corresponding to each of the plurality of initial paths is blocked by the obstacle.
The method according to claim 4, wherein the tracking is selected from the plurality of initial paths based on the cost that the observation line of sight corresponding to each of the plurality of initial paths is blocked by the obstacle. Path, including:

The initial path with the smallest corresponding cost among the plurality of initial paths is determined as the tracking path.
The method according to claim 2, wherein the generating initial path planning information at least according to the movement information of the target comprises:

Determine multiple initial target points at least according to the motion information of the target;

The determining the cost of obstructing the observation line of sight by the obstacle according to the position of the obstacle and the initial path planning information includes:

Determine the cost of obstructing the observation line of sight corresponding to each of the multiple initial target points by the obstacle according to the position of the obstacle;

The planning to obtain the tracking path based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information includes:

Based on the cost that the observation line of sight corresponding to each of the plurality of initial target points is blocked by the obstacle, the tracking path is obtained by planning.
The method according to claim 6, wherein the planning to obtain the tracking path based on the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle comprises:

Selecting a target point with the smallest corresponding cost from the multiple candidate target points;

Plan the tracking path according to the target point with the least cost.
The method according to claim 6, wherein the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle comprises:

Determine each sampling point on the line between the starting point and the initial target point of the tracking device according to the position of the obstacle;

Calculating the cost of obstructing the observation line of sight when the tracking device is at the sampling point;

The costs corresponding to the multiple sampling points are integrated to obtain the costs at which the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle.
The method according to claim 6, wherein the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle comprises:

Determining the distance between the line between the initial target point and the target and the obstacle;

According to the distance between the line and the obstacle, and the mapping relationship between the preset distance and the cost, it is obtained that the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle cost.
The method according to claim 2 or 3, wherein the planning of the tracking path is also based on obstacle avoidance cost and/or distance cost, and the obstacle avoidance cost is used to indicate the collision between the tracking device and the obstacle. Possibly, the distance cost is used to indicate the difference between the tracking parameter of the tracking device and the desired tracking parameter.
The method according to claim 10, wherein the desired tracking parameter is a preset value or input by a user of the tracking device.
The method according to claim 11, wherein the tracking parameters include one or more of the following parameters: tracking distance, tracking height, and tracking angle.
The method according to claim 1, wherein the method further comprises:

Acquiring movement information of the tracking device;

The planning the tracking path of the tracking device to the target according to the movement information of the target and the position of the obstacle includes:

Planning the tracking path of the tracking device to the target according to the movement information of the target, the location of the obstacle, and the movement information of the tracking device.
The method according to any one of claims 1-13, wherein the motion information includes one or more of the following information: position information, speed information, acceleration information, orientation information, and posture information.
The method according to any one of claims 1-13, wherein the obstacle comprises an obstacle around the tracking device and/or an obstacle around the target.
The method according to any one of claims 1-13, wherein the obtaining the position of the obstacle in the environment where the target is located comprises:

Find the location of the obstacle in the map data and/or obtain the location of the obstacle based on the sensor of the tracking device.
A tracking device, characterized in that it comprises:

Memory, used to store programs;

The processor is configured to execute the program stored in the memory to perform the following operations:

Acquiring movement information of the target and the location of obstacles in the environment where the target is located, wherein the movement information is acquired based on a visual sensor of a tracking device;

According to the movement information of the target and the position of the obstacle, the tracking path of the tracking device to the target is planned, so that when the tracking device moves along the tracking path, the target remains in the visual field. Within the observation range of the sensor.
The tracking device according to claim 17, wherein the planning a tracking path of the tracking device to the target according to the movement information of the target and the position of the obstacle comprises:

Generating initial path planning information at least according to the movement information of the target;

Determining, according to the position of the obstacle and the initial path planning information, the cost of the observing line of sight of the tracking device to the target being blocked by the obstacle;

The tracking path is planned based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information.
The method according to claim 18, wherein the cost is determined based on distance information between the obstacle and the observation line of sight.
The tracking device according to claim 18, wherein the generating initial path planning information at least according to the motion information of the target comprises:

Generate multiple initial paths at least according to the movement information of the target;

The determining the cost of obstructing the observation line of sight by the obstacle according to the position of the obstacle and the initial path planning information includes:

Determine the cost of obstructing the observation line of sight corresponding to each of the multiple initial paths by the obstacle according to the position of the obstacle;

The planning to obtain the tracking path based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information includes:

The tracking path is selected from the plurality of initial paths based on the cost that the observation line of sight corresponding to each of the plurality of initial paths is blocked by the obstacle.
The tracking device according to claim 20, wherein the observation line of sight corresponding to each of the plurality of initial paths is selected from the plurality of initial paths at the cost of being blocked by the obstacle. Tracking path, including:

The initial path with the smallest corresponding cost among the plurality of initial paths is determined as the tracking path.
The tracking device according to claim 18, wherein the generating initial path planning information at least according to the motion information of the target comprises:

Determine multiple initial target points at least according to the motion information of the target;

The determining the cost of obstructing the observation line of sight by the obstacle according to the position of the obstacle and the initial path planning information includes:

Determine the cost of obstructing the observation line of sight corresponding to each of the multiple initial target points by the obstacle according to the position of the obstacle;

The planning to obtain the tracking path based on the cost of the observation line of sight being blocked by the obstacle and the initial path planning information includes:

Based on the cost that the observation line of sight corresponding to each of the plurality of initial target points is blocked by the obstacle, the tracking path is obtained by planning.
The tracking device according to claim 22, wherein the planning to obtain the tracking path based on the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle comprises:

Selecting a target point with the smallest corresponding cost from the multiple candidate target points;

Plan the tracking path according to the target point with the least cost.
The tracking device according to claim 22, wherein the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle comprises:

Determine each sampling point on the line between the starting point and the initial target point of the tracking device according to the position of the obstacle;

Calculating the cost of obstructing the observation line of sight when the tracking device is at the sampling point;

The costs corresponding to the multiple sampling points are integrated to obtain the costs at which the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle.
The tracking device according to claim 22, wherein the determining the cost of the observation line of sight corresponding to each of the plurality of initial target points being blocked by the obstacle according to the position of the obstacle comprises:

Determining the distance between the line between the initial target point and the target and the obstacle;

According to the distance between the line and the obstacle, and the mapping relationship between the preset distance and the cost, it is obtained that the observation line of sight corresponding to each of the multiple initial target points is blocked by the obstacle cost.
The tracking device according to claim 18, wherein the planning of the tracking path is also based on an obstacle avoidance cost and/or a distance cost, and the obstacle avoidance cost is used to indicate the possibility of a collision between the tracking device and an obstacle The distance cost is used to indicate the difference between the tracking parameter of the tracking device and the desired tracking parameter.
The tracking device according to claim 26, wherein the desired tracking parameter is a preset value or input by a user of the tracking device.
The tracking device according to claim 27, wherein the tracking parameters comprise one or more of the following parameters: tracking distance, tracking height, and tracking angle.
The tracking device according to claim 17, wherein the processor is further configured to perform the following operations:

Acquiring movement information of the tracking device;

The planning the tracking path of the tracking device to the target according to the movement information of the target and the position of the obstacle includes:

Planning the tracking path of the tracking device to the target according to the movement information of the target, the location of the obstacle, and the movement information of the tracking device.
The tracking device according to any one of claims 17-29, wherein the motion information includes one or more of the following information: position information, speed information, acceleration information, orientation information, and posture information.
The tracking device according to any one of claims 17-29, wherein the obstacle comprises an obstacle around the tracking device and/or an obstacle around the target.
The tracking device according to any one of claims 17-29, wherein the acquiring the position of the obstacle in the environment where the target is located comprises:

Find the location of the obstacle in the map data and/or obtain the location of the obstacle based on the sensor of the tracking device.
A machine-readable storage medium, characterized in that instructions for executing the method according to any one of claims 1-16 are stored thereon.
A computer program product, characterized by comprising instructions for executing the method according to any one of claims 1-16.