WO2021068210A1

WO2021068210A1 - Method and apparatus for monitoring moving object, and computer storage medium

Info

Publication number: WO2021068210A1
Application number: PCT/CN2019/110674
Authority: WO
Inventors: 李延召
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-10-11
Filing date: 2019-10-11
Publication date: 2021-04-15
Also published as: CN112956187A

Abstract

Provided are a method and apparatus for monitoring a moving object, and a computer storage medium. The method comprises: acquiring detection data obtained by a lidar by means of detection (S110); determining a foreground point cloud of each frame according to background point cloud data and detection data of each frame, wherein the background point cloud data and the detection data are for the same scene (S120); and according to foreground point clouds of multiple frames, constructing the trajectory of the movement of a foreground object (S130). It can be seen that embodiments of the present invention use point cloud data detected by a lidar to construct background point cloud data, and on the basis of same, determine a moving foreground object and determine the trajectory thereof. The embodiments of the present invention may use a lidar to achieve the motion detection of a foreground object. Moreover, the algorithm complexity of the method is low, the difficulty of implementation is small, the obtained motion detection results are highly accurate, the method is easy to apply, may be applied in various scenes, for example, the method will not be affected by light and the like, and the scene adaptability is strong.

Description

Method, device and computer storage medium for monitoring moving objects

Technical field

The embodiments of the present invention relate to the field of radar, and more specifically, to a method, device, and computer storage medium for monitoring moving objects.

Background technique

At present, when monitoring moving objects, camera-based monitoring is usually adopted. However, the images collected by the camera generally lack depth information or the accuracy of the estimated depth information is poor, resulting in inaccurate monitoring results. In addition, the use of cameras for monitoring has poor environmental adaptability, especially in environments that are easily affected by light.

Summary of the invention

The embodiments of the present invention provide a method, a device and a computer storage medium for monitoring a moving object, which can determine the trajectory of the movement of a foreground object and can be applied to various scenes.

In the first aspect, a method for monitoring moving objects is provided, including:

Obtain detection data obtained by lidar through detection;

Determine the front scenic spot cloud of each frame according to the background point cloud data and the detection data of each frame, wherein the background point cloud data and the detection data are for the same scene;

According to the front scenic spot cloud of multiple frames, the trajectory of the foreground object is constructed.

In a second aspect, a device for monitoring moving objects is provided, including:

The acquisition module is used to acquire the detection data obtained by the lidar through detection;

The determining module is configured to determine the front sight cloud of each frame according to the background point cloud data and the detection data of each frame, wherein the background point cloud data and the detection data are for the same scene;

The construction module is used to construct the trajectory of the foreground object according to the front sight cloud of multiple frames.

In a third aspect, a device for monitoring a moving object is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the foregoing when the computer program is executed. The steps of the method described in the first aspect.

In a fourth aspect, a computer storage medium is provided with a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

It can be seen that, in the embodiment of the present invention, the point cloud data detected by the lidar is used to construct the background point cloud data, and on this basis, the moving foreground object is determined, and the moving trajectory of the foreground object is determined. The embodiment of the present invention can realize the motion detection of foreground objects through lidar. Moreover, the method of the embodiment of the present invention has low algorithm complexity and is not difficult to implement. The obtained motion detection result has high accuracy and is easy to apply. , And can be applied to various scenes, for example, it will not be affected by light, etc., and the scene adaptability is strong.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, without creative labor, other drawings can be obtained based on these drawings.

FIG. 1 is a schematic flowchart of a method for monitoring a moving object according to an embodiment of the present invention;

2 is a schematic diagram of the spatial distribution of the depth value of the background according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of a front scenic spot cloud according to an embodiment of the present invention;

4 is a schematic diagram of foreground objects determined according to the cloud of front scenic spots according to an embodiment of the present invention;

Fig. 5 is a schematic block diagram of a mobile object monitoring device according to an embodiment of the present invention;

FIG. 6 is another schematic block diagram of a device for monitoring a moving object according to an embodiment of the present invention;

Fig. 7 is another schematic block diagram of a mobile object monitoring device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

With the significant reduction in the cost of lidar, its application is bound to become increasingly widespread. The embodiment of the present invention provides a solution for monitoring moving objects based on lidar.

As shown in FIG. 1, it is a schematic flowchart of a method for monitoring a moving object according to an embodiment of the present invention. The method shown in Figure 1 includes:

S110: Obtain detection data obtained by lidar through detection;

S120: Determine the front scenic spot cloud of each frame according to the background point cloud data and the detection data of each frame, wherein the background point cloud data and the detection data are for the same scene;

S130: Construct a moving trajectory of the foreground object according to the front scenic spot cloud of multiple frames.

Among them, for different scenes, the moving objects can also be different. As an example, if the scene is a parking lot, the moving object may be a vehicle, for example, a car, a bicycle, and so on. As another example, if the scene is a road, the moving objects may be vehicles, or the moving objects may be vehicles and pedestrians. Those skilled in the art can understand that the scene can be other, and correspondingly, the moving object can be other moving objects in the scene, for example, the scene is water, the moving object is fish, and so on. The present invention will not enumerate one by one.

The embodiment of the present invention mainly takes a vehicle as an example for detailed explanation, and the corresponding scene is a location where the vehicle may appear, such as a parking lot, a road, and the like.

Exemplarily, before S110, it may include: S101, constructing background point cloud data. This process can be called a background modeling process, and the background point cloud data can also be called a background point cloud pool.

Specifically, the background point cloud data can be obtained in the following manner: acquiring the background detection data detected by the lidar over a period of time; constructing the background point cloud data according to the detected background detection data.

Wherein, the length of a period of time can be preset according to the scene, for example, 24 hours or 2 days or other length of time, which is not limited in the present invention.

Among them, during initialization, after the lidar is fixed, the lidar can be turned on for a period of time, and sufficient detection data within this period of time can be collected to determine that all backgrounds in the background are detected and collected. Optionally, if there is a movable object currently in a stationary state in the scene, the object should be removed at a certain or certain moments during this period of time, so as to ensure that the collection of complete background information is included. Background detection data. For example, if there is a stationary vehicle in the scene, it should be ensured that it has been removed during this period of time.

Among them, constructing the background point cloud data according to the background detection data may include: projecting the scanning points of the background detection data on the projection surface and gridding the projection surface; for each grid on the projection surface, determining the projection to the grid The background feature of the scanned point on the Exemplarily, the background feature of the scanning point can be determined according to the depth information and/or reflectivity of the corresponding scanning point of each frame of background detection data.

Optionally, the projection surface may be a plane perpendicular to the axis of the radar, of course, the projection surface may also be other planes, such as a certain angle with the axis of the radar. Subsequently, the projection surface can be gridded (also referred to as discretization), and each grid on the gridded projection surface has scan points projected on it.

Further, the background feature can be determined based on depth information and/or reflectivity. Taking depth information as an example, for a certain scanning point, the depth information may include a depth value, and the point with the largest depth value may be selected as its background feature. Alternatively, the depth information may include the distribution density function of the depth value, and the distribution density function represents the probability density of the depth value, which represents the probability density of different depth values being determined as the background. The distribution density function of its depth value can be used as its background feature.

As an example, FIG. 2 shows a schematic diagram of the spatial distribution of the depth value of the background. The scene in Fig. 2 is the truth, such as a highway; where the sky above the picture, the picture includes roads, road signs, street lights, etc., and trees beside the road.

In addition, in S101, a process of in-plane filtering may also be included. Specifically, a filtering method can be used to reduce the influence of noise on the background features of the scanned points in the background point cloud data. Among them, the filtering method may be average filtering or other filtering methods.

Optionally, the embodiment of the present invention may also update the background point cloud data based on the detection data in real time or periodically. Specifically, after the background point cloud data is constructed, it is updated according to the subsequent detection data. For example, if the depth of a certain grid point in the subsequent detection data is greater than the depth of the corresponding grid point in the background point cloud data, it means that the background depth of the point has changed, and the point can be determined based on the detection data. The background features are updated.

In this way, in S101, background point cloud data can be constructed by means of background modeling. It can be used for subsequent foreground extraction to facilitate the detection of moving objects.

Exemplarily, S110 may include: real-time acquisition by lidar to obtain detection data, where the lidar is not moved after being fixed, that is, the scene collected by lidar in S110 and the background point cloud constructed in S101 The scenes collected by lidar are the same when data is collected.

Exemplarily, S120 may include: projecting the scanning point of the detection data onto the projection surface to obtain the projection point; by comparing the projection point of the scanning point with the background projection point of the background point cloud data at the corresponding position of the projection surface, determining the scanning Whether the point belongs to the former scenic spot cloud.

It can be understood that the projection surface selected when constructing the cloud data of the front scenic spot in S101 is the same projection surface as the projection surface here, for example, both are planes perpendicular to the axis of the radar. In addition, the method of gridding (or discretizing) the projection surface is also the same.

The projection of the detection data detected in S110 on the projection surface is called the projection point, and the projection of the background point cloud data in S101 on the projection surface is called the background projection point. When judging whether a certain scanning point in the detection data belongs to the front scenic spot cloud, it can be judged by comparing the projection point with the background projection point. Specifically, if the absolute value of the difference between the projection point and the background projection point is greater than or equal to the first preset threshold, it is determined that the scan point belongs to the front scenic spot cloud; If the absolute value of the difference between the background projection points is less than the first preset threshold, it is determined that the scanning point does not belong to the front scenic spot cloud.

Wherein, the calculated difference value may be the difference value of the attribute value between the pixel points, and the attribute value may be depth, reflectance or other values. Wherein, the first preset threshold can be set according to the attributes of the radar, the attributes of the scene, and so on.

That is to say, if the absolute value of the difference in the attribute value between the projection point of a certain scan point and the corresponding background projection point in the detection data is greater than the first preset threshold, the scan point is determined to be the front scenic spot; otherwise As the background point.

In this way, through this process, for each frame of detection data detected by the lidar, it is possible to obtain which of the previous scenic spots in each frame are. Furthermore, the previous scenic spot cloud can be obtained by aggregating the previous scenic spots.

Among them, the collection of all the front scenic spots in a frame can be constructed as the front scenic spot cloud of this frame. As shown in Fig. 3, it is a schematic diagram of the front scenic spot cloud of one frame.

Further, the foreground object in each frame can be determined according to the front sight cloud of each frame. As shown in FIG. 4, different gray levels are used to represent different foreground objects, including foreground object 1 to foreground object 6.

Specifically, the front scenic spot cloud whose distance is less than the second preset threshold can be regarded as a point cloud cluster, which belongs to the same foreground object, that is, the foreground object contains the distance between every two adjacent front scenic spots. The distance is less than the second preset threshold. The second preset threshold may be preset according to the attributes of the radar, for example, according to the density of the points detected by the radar.

Further, the geometric attributes of the foreground object can be determined according to all the front scenic spot clouds contained in the foreground object.

Optionally, the geometric attributes may include at least one of the following: the size, center coordinates, and center of gravity coordinates of the foreground object. Exemplarily, the size is the side length or area of a bounding box of all front scenic spot clouds contained in the foreground object. As another example, the size may be the difference between the maximum value and the minimum value of the coordinates of all points in the front scenic spot cloud included in the foreground object. Exemplarily, the center coordinates are the coordinates of the center points of all front scenic spot clouds contained in the foreground object or the coordinates of the center point of the bounding box. Exemplarily, the center of gravity coordinates are the coordinates of the center of gravity of all the front scenic spot clouds contained in the foreground object or the coordinates of the center of gravity of the bounding box.

Further, it may also include: identifying the type of the foreground object according to the front scenic spot cloud contained in the foreground object. For example, the type of foreground object can be determined according to the shape of the outer contours of all front scenic spot clouds contained in the foreground object. For example, the types can include cars, bicycles, pedestrians, and so on.

It can be seen that for one frame of detection data detected by lidar, it is possible to determine which points are front sights, construct a cloud of front sights, determine foreground objects based on the cloud of front sights, and further determine the type and geometric properties of the foreground objects. Among them, the center coordinates or the center of gravity coordinates in the geometric attributes can be used as the position information of the foreground object. A similar process is performed for each frame, and the foreground objects in each frame can be obtained.

Exemplarily, S130 may include: predicting the predicted position information of the foreground object in the current frame according to the position information of the foreground object in the historical frame; and determining that the foreground object is in the current frame according to the predicted position information of the foreground object in the current frame The actual position information of the foreground object; according to the position information of the foreground object in the historical frame and the actual position information of the foreground object in the current frame, the trajectory of the foreground object's movement is constructed.

It can be understood that there may be multiple foreground objects in one frame, so it is necessary to determine which of the same foreground objects in different frames are, and then the movement trajectory of a certain foreground object can be determined.

Optionally, based on the historical trajectory constructed by the position information of the foreground object in the historical frame, the predicted position information of the foreground object in the current frame can be predicted by fitting.

Assume that the current frame is the t+1th frame. For example, for the foreground object A, the predicted position information of the t+1th frame can be predicted according to the position information of each frame from the 0th frame (or the t-nth frame) to the tth frame. Predictive models can be used to make predictions. As an example, the predictive model may be a machine learning model. As another example, the prediction model may be a function model obtained by curve fitting historical location information.

Optionally, determine the position information of each moving object in the current frame; determine which of each moving object is the foreground according to the predicted position information of the foreground object in the current frame and the position information of each moving object in the current frame Object and get the actual position information of the foreground object in the current frame.

That is to say, the position information of each moving object (ie foreground object) in the t+1th frame can be determined according to the actually detected detection data of the t+1th frame, and the t+1th frame can be determined by position matching Which foreground object in is foreground object A.

Specifically, the distance difference between the position information of each moving object and the predicted position information can be calculated; the absolute value of the distance difference that is less than the third preset threshold and the smallest is the absolute value of the distance difference corresponding to the movement The object is determined to be the foreground object.

If the position information of each moving object is actually detected in the t+1 frame, the distance between the position of several moving objects and the predicted position of the foreground object A is less than the third preset threshold, then the distance The smallest moving object is determined to be the foreground object A.

In addition, as described above, the position information is center coordinates or center of gravity coordinates. Then, the center coordinate can be used as the position information to calculate the distance difference, or the center of gravity coordinate can be used as the position information to calculate the distance difference. Generally, these two calculation results are different. At this time, the foreground object can be matched based on the smallest distance difference.

For example, suppose that m moving objects are detected in the t+1th frame. If the center coordinates are used as the position information, the m distance differences between the position information of the m moving objects and the predicted position information of the predicted foreground object A can be calculated. If the center of gravity coordinates are used as the position information, the m distance differences between the position information of the m moving objects and the predicted position information of the predicted foreground object A can also be calculated. In other words, a total of 2*m distance differences can be obtained. If the minimum value of these 2*m distance differences is obtained by using the center coordinates as the position information, the center coordinates are used as the position information to determine which one of the m moving objects is the foreground object A. On the contrary, if the minimum value of the 2*m distance difference values is obtained by using the center of gravity coordinates as the position information, the center of gravity coordinates are used as the position information to determine which one of the m moving objects is the foreground object A.

In addition, if the distance difference between the position information of each moving object in the current frame and the predicted position information is greater than or equal to the third preset threshold, it is determined that the foreground object is not found in the current frame. In other words, the foreground object is not detected in the current frame, and its trajectory is interrupted.

Returning to the above example, if the absolute values of the m distance differences between the position information of m moving objects and the predicted position information of the predicted foreground object A are greater than or equal to the third preset threshold, then m moving objects are described Each of them is not foreground A.

For this situation, that is, if the foreground object A is not found in the t+1 frame, if the position information of the foreground object A in the t frame is not at the edge of the radar's field of view, it can be further detected in the subsequent radar Detect whether there is foreground object A in the matching data. Assume that the current frame is the t+pth frame. For the foreground object A, the predicted position information of the t+pth frame can be predicted based on the position information of each frame from the 0th frame (or the t-nth frame) to the tth frame. And can determine the position information of each moving object (ie foreground object) in the t+p frame according to the actually detected detection data of the t+pth frame, and determine which of the t+pth frame is through position matching The foreground object is foreground object A.

For example, suppose that m1 moving objects are detected in the t+pth frame. The m1-1 moving objects in the m1 moving objects are matched with the foreground objects in the t+p-1 frame, and the m1 moving object in the m1 moving objects cannot be in the t+p-1 frame Find a matching foreground object. At this time, it can be judged based on the predicted position information of the foreground object A in the t+p frame and the detected position information of the m1 moving object among the m1 moving objects in the t+p frame Whether the m1 moving object among the m1 moving objects is the foreground object A.

Specifically, if the detected position information of the m1-th moving object in the t+p-th frame among the m1 moving objects is not located at the edge of the field of view detected by the radar, the judgment process is performed.

It can be seen that, in the embodiment of the present invention, the position information of each foreground object can be determined based on the detection data detected by the radar, and the position information of the foreground object in each frame can be determined by matching between different frames, so that the position information of each foreground object can be determined according to each frame To construct the trajectory of the foreground object’s movement. Among them, the trajectory can characterize the change of position information over time.

In the embodiment of the present invention, the information of the trajectory of the movement of the foreground object may be stored. Exemplarily, the trajectory of the foreground object may be stored in the form of a list, wherein the list records the position information of the foreground object at each time. As an example, the list may have the form of Table 1 as shown below.

Table I

It should be noted that the embodiment of the present invention does not limit the form and content of the list. For example, the list may be in the form of a stack. For example, the center position and the center of gravity position may be recorded in the list at the same time, and so on. The present invention is not limited to this.

Exemplarily, in S130, point clouds in different frames may be matched to determine which ones belong to the same foreground object. Further, it is also possible to accumulate the front sight clouds of the foreground object in multiple frames. For example, if the number of point clouds included in the foreground object in a certain frame is n1, and the number of point clouds included in another frame is n2, then the point clouds in the two frames can be accumulated, specifically according to the foreground The shape and posture of the object are accumulated, assuming that the number of point clouds after accumulation is n3, n3≥n1, n3≥n2 and n3≤n1+n2. In this way, a denser point cloud of the foreground object can be obtained, so that the foreground object can be better characterized, and on the other hand, the type of the foreground object can be verified in turn, which can improve the accuracy of monitoring.

Further, after S130, it may further include: determining a movement parameter of the foreground object according to the trajectory of the movement of the foreground object, where the movement parameter includes at least one of the following: displacement, velocity, and acceleration.

Specifically, the movement parameters of the foreground object in a certain period of time can be determined according to the trajectory. As an implementation manner, the movement parameter may also be included in the above list.

Assuming that the foreground object is a vehicle, it can be determined whether the vehicle is in an overspeeding state according to the movement parameters of the foreground object. Specifically, the speed of the vehicle can be determined, and the speed can be compared with the speed limit in the current scene to determine whether the vehicle is in an overspeeding state. For example, the lidar can be installed at the signal light at the intersection of the road, so as to detect whether the vehicle passing through the intersection is speeding, and assist in the violation monitoring.

As another example, even if the current speed of the vehicle is not overspeeding, it can be predicted whether it is likely to be overspeeding based on the current speed and acceleration of the vehicle. If so, the traffic controller or the driver can be warned or reminded in advance to ensure traffic safety and prevent traffic accidents due to speeding.

Further, assuming that the foreground object is a vehicle, after S130, it may further include: performing statistics on traffic flow information in a specific area. Specifically, the traffic flow information in the detected scene can be counted, and based on the result of the statistics, it can be determined whether there is congestion or other conditions.

Further, assuming that the foreground object is a vehicle and the scene is a parking lot, after S130, it may further include: determining the parking space occupancy status of the parking lot according to the detected movement trajectory of the foreground object in the parking lot. Specifically, it can be determined according to the trajectory how many vehicles have entered the parking lot, and it can be determined which parking spaces have been occupied by these vehicles that have entered, and which parking spaces are still free. Subsequently, when another vehicle to be parked enters, the vehicle to be parked can be guided to a free parking space. In this way, intelligent parking and automated management of parking lots can be realized by means of automatic driving (or unmanned driving).

According to another embodiment, FIG. 5 is a schematic block diagram of an apparatus for monitoring a moving object according to an embodiment of the present invention. The apparatus 20 shown in FIG. 5 includes an acquisition module 210, a determination module 220, and a construction module 230.

The acquiring module 210 is used to acquire the detection data obtained by the lidar through detection;

The determining module 220 is configured to determine the front scenic spot cloud of each frame according to the background point cloud data and the detection data of each frame, wherein the background point cloud data and the detection data are for the same scene;

The construction module 230 is used for constructing a trajectory of the foreground object's movement according to the front scenic spot cloud of multiple frames.

Exemplarily, as shown in FIG. 6, the device 20 may further include a background data acquisition module 201 and a background point cloud construction module 202. The background data acquisition module 201 may be used to acquire background detection data detected by the lidar over a period of time. The background point cloud construction module 202 may be used to construct the background point cloud data according to the background detection data.

Exemplarily, the background point cloud construction module 202 may include a projection sub-module and a feature determination sub-module. The projection sub-module is used to project the scan points of the background detection data onto the projection surface and the projection surface is gridded; the feature determination sub-module is used to determine for each grid on the projection surface The background features of the scanned points projected onto the grid.

Optionally, the feature determination submodule may be specifically configured to determine the background feature of the scan point according to the depth information and/or reflectivity of the corresponding scan point of each frame of background detection data.

Wherein, the depth information includes a distribution density function of the depth value, and the distribution density function represents the probability density of the depth value. Specifically, it indicates the probability density of different depth values being judged as background

Exemplarily, the background point cloud construction module 202 may also be used to reduce the influence of noise on the background features of the scan points in the background point cloud data by adopting a filtering method.

Exemplarily, the device 20 may further include a background point cloud update module, configured to update the background point cloud data according to the detection data in real time or periodically. Specifically, after constructing the background point cloud data, it is updated based on the subsequent detection data. For example, if the depth of a certain grid point in the subsequent detection data is greater than the depth of the corresponding grid point in the background point cloud data, it means that the background depth of the point has changed, and the point can be determined based on the detection data. The background features are updated.

Exemplarily, as shown in FIG. 6, the determining module 220 may include a projection unit 2201 and a judgment unit 2202.

The projection unit 2201 can be used to project the scanning point of the detection data onto a projection surface to obtain a projection point; the judging unit 2202 can be used to compare the projection point of the scanning point with the background point cloud data in the The background projection points at the corresponding positions of the projection surface are compared, and it is judged whether the scanning point belongs to the front scenic spot cloud.

Optionally, the judging unit 2202 may be specifically configured to: if the absolute value of the difference between the projection point and the background projection point is greater than or equal to a first preset threshold, determine that the scan point belongs to the front scenic spot cloud If the absolute value of the difference between the projection point and the background projection point is less than the first preset threshold, it is determined that the scan point does not belong to the front scenic spot cloud.

Exemplarily, the determining module 220 may also be used to determine the foreground object according to the cloud of front sights, wherein the distance between every two adjacent front sights contained in the foreground object is less than the second predicted Set a threshold; determine the geometric attributes of the foreground object according to all front sight clouds included in the foreground object.

Wherein, the geometric attributes include at least one of the following: the size, center coordinates, and center of gravity coordinates of the foreground object.

Wherein, the size is the side length or area of the bounding box of all the front scenic spot clouds. The center coordinates are the coordinates of the center points of all the front scenic spot clouds or the coordinates of the center point of the bounding box. The center of gravity coordinates are the coordinates of the center of gravity of all the front scenic spot clouds or the coordinates of the center of gravity of the bounding box.

Exemplarily, as shown in FIG. 6, the construction module 230 may include a prediction unit 2301, a determination unit 2302 and a construction unit 2303.

The prediction unit 2301 may be configured to predict the predicted position information of the foreground object in the current frame according to the position information of the foreground object in the historical frame. The determining unit 2302 may be configured to determine the actual position information of the foreground object in the current frame according to the predicted position information of the foreground object in the current frame. The constructing unit 2303 may be configured to construct a trajectory of the foreground object's movement according to the position information of the foreground object in the historical frame and the actual position information of the foreground object in the current frame.

Optionally, the prediction unit 2301 may be specifically configured to predict the predicted position information of the foreground object in the current frame by fitting a historical trajectory constructed based on the position information of the foreground object in the historical frame.

Optionally, the determining unit 2302 may be specifically configured to: determine the position information of each moving object in the current frame; according to the predicted position information of the foreground object in the current frame and each moving object in the current frame To determine which of the moving objects is the foreground object and obtain the actual position information of the foreground object in the current frame.

Optionally, the determining unit 2302 may be specifically configured to: calculate the distance difference between the position information of each moving object and the predicted position information; and set the absolute value of the distance difference to be less than a third preset threshold. And the moving object corresponding to the absolute value of the smallest distance difference is determined as the foreground object.

Optionally, the determining unit 2302 may also be specifically configured to: if the absolute value of the distance difference is greater than or equal to the third preset threshold, determine that the foreground object is not found in the current frame.

Exemplarily, as shown in FIG. 6, the device 20 may further include a storage module 240, configured to store the trajectory of the foreground object in the form of a list, wherein the list records the status of the foreground object at each time. location information.

Wherein, the position information includes: the coordinates of the center point or the coordinates of the center of gravity point.

Exemplarily, the device 20 may further include a parameter determination module, configured to determine a movement parameter of the foreground object according to the trajectory of the movement of the foreground object, wherein the movement parameter includes at least one of the following: displacement, Speed, acceleration.

Exemplarily, as shown in FIG. 6, the device 20 further includes a post-processing module 250.

As an example, the post-processing module 250 may be used to accumulate the foreground cloud of the foreground object in multiple frames.

As an example, the post-processing module 250 may be used to identify the type of the foreground object according to the front sight cloud included in the foreground object. Among them, the types include: cars, bicycles, and pedestrians.

As an example, the foreground object is a vehicle, and the post-processing module 250 may be used to determine whether the vehicle is in an overspeeding state.

As an example, the foreground object is a vehicle, and the post-processing module 250 may be used to perform statistics on traffic flow information in a specific area.

As an example, the scene is a parking lot, the foreground object is a vehicle, and the post-processing module 250 may be used to determine the parking space occupancy status of the parking lot according to the detected movement trajectory of the foreground object in the parking lot. . Optionally, the post-processing module 250 may be further used to guide the vehicle to be parked to an idle parking space.

The device 20 shown in FIG. 5 or FIG. 6 can implement the aforementioned method for monitoring a moving object shown in FIG. 1. To avoid repetition, it will not be repeated here.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

In addition, the embodiment of the present invention also provides another device for monitoring a moving object, including a memory, a processor, and a computer program stored in the memory and running on the processor. The processor executes the program when the program is executed. The steps of the method for monitoring a moving object shown in FIG. 1 described above.

As shown in FIG. 7, the device 30 may include a memory 310 and a processor 320. The memory 310 stores computer program codes for implementing corresponding steps in the method for monitoring a moving object according to an embodiment of the present invention. The processor 320 is configured to run the computer program code stored in the memory 310 to execute the corresponding steps of the method for monitoring a moving object according to an embodiment of the present invention, and to implement the steps described in FIG. 5 or FIG. 6 according to the embodiment of the present invention. The various modules in the device 20.

Exemplarily, when the computer program code is executed by the processor 320, the following steps are executed: acquiring detection data obtained by the lidar through detection; determining the front spot of each frame according to the background point cloud data and the detection data of each frame Cloud, wherein the background point cloud data and the detection data are for the same scene; according to multiple frames of the front scenic spot cloud, the trajectory of the foreground object is constructed.

In addition, the embodiment of the present invention also provides a computer storage medium on which a computer program is stored. When the computer program is executed by the processor, the steps of the method for monitoring a moving object shown in FIG. 1 can be implemented. For example, the computer storage medium is a computer-readable storage medium.

The computer storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory ( CD-ROM), USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media. For example, one computer-readable storage medium contains computer-readable program code for monitoring moving objects, and another computer-readable storage medium contains computer-readable storage media based on Computer-readable program code for the post-processing of the trajectory.

In addition, the embodiment of the present invention also provides a computer program or computer program product. The computer program or computer program product can be executed by a processor, and when the code is executed by the processor, it can realize: Detection data; according to the background point cloud data and the detection data of each frame, determine the front sight cloud of each frame, wherein the background point cloud data and the detection data are for the same scene; according to the front sight cloud of multiple frames, Construct the trajectory of the foreground object's movement.

It can be seen that, in the embodiment of the present invention, the point cloud data detected by the lidar is used to construct the background point cloud data, and on this basis, the moving foreground object is determined, and the moving trajectory of the foreground object is determined. The embodiment of the present invention can realize the motion detection of foreground objects through lidar. Moreover, the method of the embodiment of the present invention has low algorithm complexity and is not difficult to implement. The obtained motion detection result has high accuracy and is easy to apply. , And can be applied to various scenes, for example, it will not be affected by light, etc., and the scene adaptability is strong. Further, the embodiment of the present invention can also determine parameters such as the speed and acceleration of the moving object based on the trajectory of the moving object, and can also perform statistics on the traffic information of the moving object. This simple and intelligent statistics can also become an important part of the construction of smart cities.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc. .

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, each functional unit in each embodiment of the present application may be integrated in a processor, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for monitoring a moving object, which is characterized in that it includes:

Obtain detection data obtained by lidar through detection;

Determine the front scenic spot cloud of each frame according to the background point cloud data and the detection data of each frame, wherein the background point cloud data and the detection data are for the same scene;

According to the front scenic spot cloud of multiple frames, the trajectory of the foreground object is constructed.
The method according to claim 1, characterized in that, before the acquiring the detection data, it further comprises acquiring the background point cloud data in the following manner:

Acquiring background detection data detected by the lidar over a period of time;

The background point cloud data is constructed according to the background detection data.
The method according to claim 2, wherein the constructing the background point cloud data according to the background detection data comprises:

Projecting the scanning points of the background detection data onto a projection surface and gridding the projection surface;

For each grid on the projection surface, the background feature of the scanning point projected on the grid is determined.
The method according to claim 3, wherein determining the background characteristics of the scanning points projected on the grid comprises:

According to the depth information and/or reflectivity of the corresponding scanning point of each frame of background detection data, the background feature of the scanning point is determined.
The method according to claim 4, wherein the depth information comprises a distribution density function of a depth value, and the distribution density function represents a probability density of the depth value.
The method according to any one of claims 2-5, wherein the process of constructing the background point cloud data further comprises:

A filtering method is used to reduce the influence of noise on the background features of the scanned points in the background point cloud data.
The method according to any one of claims 1-6, further comprising:

The background point cloud data is updated in real time or periodically according to the detection data.
The method according to any one of claims 1-7, wherein the determining the front sight cloud of each frame comprises:

Projecting the scanning point of the detection data onto the projection surface to obtain the projection point;

By comparing the projection point of the scanning point with the background projection point of the background point cloud data at the corresponding position of the projection surface, it is determined whether the scanning point belongs to the front scenic spot cloud.
The method according to claim 8, wherein the said projection point of the scanning point is compared with the background projection point of the background point cloud data at the corresponding position of the projection surface to determine the State whether the scan point belongs to the former scenic spot cloud, including:

If the absolute value of the difference between the projection point and the background projection point is greater than or equal to a first preset threshold, determining that the scanning point belongs to the front scenic spot cloud;

If the absolute value of the difference between the projection point and the background projection point is less than the first preset threshold, it is determined that the scan point does not belong to the front scenic spot cloud.
The method according to any one of claims 1-9, characterized in that, after determining the front sight cloud of each frame, the method further comprises:

Determine the foreground object according to the cloud of front sights, wherein the distance between every two adjacent front sights included in the foreground object is less than a second preset threshold;

Determine the geometric attributes of the foreground object according to all the front sight clouds included in the foreground object.
The method according to claim 10, wherein the geometric attributes include at least one of the following: the size, center coordinates, and center of gravity coordinates of the foreground object.
The method of claim 11, wherein:

The size is the side length or area of the bounding box of all front scenic spot clouds,

The center coordinates are the coordinates of the center points of the all front scenic spot clouds or the coordinates of the center point of the bounding box,

The center of gravity coordinates are the coordinates of the center of gravity of all the front scenic spot clouds or the coordinates of the center of gravity of the bounding box.
The method according to any one of claims 1-12, wherein the constructing a trajectory of the movement of the foreground object according to the front scenic spot cloud of multiple frames comprises:

Predict the predicted position information of the foreground object in the current frame according to the position information of the foreground object in the historical frame;

Determine the actual position information of the foreground object in the current frame according to the predicted position information of the foreground object in the current frame;

According to the position information of the foreground object in the historical frame and the actual position information of the foreground object in the current frame, a trajectory of the movement of the foreground object is constructed.
The method according to claim 13, wherein the predicting the predicted position information of the foreground object in the current frame according to the position information of the foreground object in the historical frame comprises:

Based on the historical trajectory constructed based on the position information of the foreground object in the historical frame, the predicted position information of the foreground object in the current frame is predicted by fitting.
The method according to claim 13 or 14, wherein the determining the actual position information of the foreground object in the current frame comprises:

Determining the position information of each moving object in the current frame;

According to the predicted position information of the foreground object in the current frame and the position information of each moving object in the current frame, determine which of the moving objects is the foreground object and obtain the foreground object in the current frame. Describe the actual position information in the current frame.
The method according to claim 15, wherein the determining which of the moving objects is the foreground object comprises:

Calculating the distance difference between the position information of each moving object and the predicted position information;

The moving object corresponding to the absolute value of the distance difference that is less than the third preset threshold and the smallest absolute value of the distance difference is determined as the foreground object.
The method according to claim 16, further comprising:

If the absolute value of the distance difference is greater than or equal to the third preset threshold, it is determined that the foreground object is not found in the current frame.
The method according to any one of claims 13-17, further comprising:

The trajectory of the foreground object is stored in the form of a list, wherein the list records position information of the foreground object at each time.
The method according to any one of claims 13-18, wherein the position information comprises: coordinates of a center point or coordinates of a center of gravity point.
The method according to any one of claims 1-19, further comprising:

According to the trajectory of the movement of the foreground object, a movement parameter of the foreground object is determined, wherein the movement parameter includes at least one of the following: displacement, velocity, and acceleration.
The method according to claim 20, wherein the foreground object is a vehicle, and the method further comprises:

Determine whether the vehicle is in a state of speeding.
The method according to any one of claims 1-21, further comprising:

Accumulate the front sight clouds of the foreground object in multiple frames.
The method according to any one of claims 1-22, further comprising:

Identify the type of the foreground object according to the front sight cloud included in the foreground object.
The method according to claim 23, wherein the types include: cars, bicycles, and pedestrians.
The method according to any one of claims 1-24, wherein the foreground object is a vehicle, and the method further comprises:

Carry out statistics on traffic information in a specific area.
The method according to any one of claims 1-24, wherein the scene is a parking lot and the foreground object is a vehicle, and the method further comprises:

Determine the parking space occupancy status of the parking lot according to the detected movement trajectory of the foreground object in the parking lot.
The method according to claim 26, wherein the method further comprises:

Guide the vehicle to be parked to a free parking space.
A device for monitoring a moving object, which is characterized in that it comprises:

The acquisition module is used to acquire the detection data obtained by the lidar through detection;

The determining module is configured to determine the front sight cloud of each frame according to the background point cloud data and the detection data of each frame, wherein the background point cloud data and the detection data are for the same scene;

The construction module is used to construct the trajectory of the foreground object according to the front sight cloud of multiple frames.
The device according to claim 28, further comprising:

A background data acquisition module, configured to acquire background detection data detected by the lidar over a period of time;

The background point cloud construction module is used to construct the background point cloud data according to the background detection data.
The device according to claim 29, wherein the background point cloud building module comprises:

A projection sub-module for projecting the scanning points of the background detection data onto a projection surface and gridding the projection surface;

The feature determination sub-module is used to determine the background feature of the scanning point projected on the grid for each grid on the projection surface.
The device according to claim 30, wherein the feature determining submodule is specifically configured to:

According to the depth information and/or reflectivity of the corresponding scanning point of each frame of background detection data, the background feature of the scanning point is determined.
The device according to claim 31, wherein the depth information comprises a distribution density function of a depth value, and the distribution density function represents a probability density of the depth value.
The device according to any one of claims 29-32, wherein the background point cloud building module is further used for:

A filtering method is used to reduce the influence of noise on the background features of the scanned points in the background point cloud data.
The device according to any one of claims 28-33, further comprising a background point cloud update module, configured to:

The background point cloud data is updated in real time or periodically according to the detection data.
The device according to any one of claims 28-34, wherein the determining module comprises:

A projection unit for projecting the scanning point of the detection data onto the projection surface to obtain a projection point;

The judging unit is configured to compare the projection point of the scan point with the background projection point of the background point cloud data at the corresponding position of the projection surface to determine whether the scan point belongs to the front sight cloud.
The device according to claim 35, wherein the judging unit is specifically configured to:

If the absolute value of the difference between the projection point and the background projection point is greater than or equal to a first preset threshold, determining that the scanning point belongs to the front scenic spot cloud;

If the absolute value of the difference between the projection point and the background projection point is less than the first preset threshold, it is determined that the scanning point does not belong to the front scenic spot cloud.
The device according to any one of claims 28-36, wherein the determining module is further configured to:

Determine the foreground object according to the front scenic spot cloud, wherein the distance between every two adjacent front scenic spots included in the foreground object is less than a second preset threshold;

Determine the geometric attributes of the foreground object according to all the front sight clouds included in the foreground object.
The device according to claim 37, wherein the geometric attributes include at least one of the following: the size, center coordinates, and center of gravity coordinates of the foreground object.
The device of claim 38, wherein:

The size is the side length or area of the bounding box of all front scenic spot clouds,

The center coordinates are the coordinates of the center points of the all front scenic spot clouds or the coordinates of the center point of the bounding box,

The center of gravity coordinates are the coordinates of the center of gravity of all the front scenic spot clouds or the coordinates of the center of gravity of the bounding box.
The device according to any one of claims 28-39, wherein the building module comprises:

A prediction unit, configured to predict the predicted position information of the foreground object in the current frame according to the position information of the foreground object in the historical frame;

A determining unit, configured to determine the actual position information of the foreground object in the current frame according to the predicted position information of the foreground object in the current frame;

The constructing unit is configured to construct a trajectory of the foreground object's movement according to the position information of the foreground object in the historical frame and the actual position information of the foreground object in the current frame.
The device according to claim 40, wherein the prediction unit is specifically configured to:

Based on the historical trajectory constructed based on the position information of the foreground object in the historical frame, the predicted position information of the foreground object in the current frame is predicted by fitting.
The device according to claim 40 or 41, wherein the determining unit is specifically configured to:

Determining the position information of each moving object in the current frame;

According to the predicted position information of the foreground object in the current frame and the position information of each moving object in the current frame, determine which of the moving objects is the foreground object and obtain the foreground object in the current frame. Describe the actual position information in the current frame.
The device according to claim 42, wherein the determining unit is specifically configured to:

Calculating the distance difference between the position information of each moving object and the predicted position information;

The moving object corresponding to the absolute value of the distance difference that is less than the third preset threshold and the smallest absolute value of the distance difference is determined as the foreground object.
The device according to claim 43, wherein the determining unit is further specifically configured to:

If the absolute value of the distance difference is greater than or equal to the third preset threshold, it is determined that the foreground object is not found in the current frame.
The device according to any one of claims 40-44, further comprising a storage module, configured to:

The trajectory of the foreground object is stored in the form of a list, wherein the list records position information of the foreground object at each time.
The device according to any one of claims 40-45, wherein the position information comprises: coordinates of a center point or coordinates of a center of gravity point.
The device according to any one of claims 28-46, further comprising a parameter determination module, configured to:

According to the trajectory of the movement of the foreground object, a movement parameter of the foreground object is determined, wherein the movement parameter includes at least one of the following: displacement, velocity, and acceleration.
The device according to claim 47, wherein the foreground object is a vehicle, and the device further comprises a post-processing module for:

Determine whether the vehicle is in an overspeeding state.
The device according to any one of claims 28-48, further comprising a post-processing module for:

Accumulate the front sight clouds of the foreground object in multiple frames.
The device according to any one of claims 28-49, further comprising a post-processing module for:

Identify the type of the foreground object according to the front sight cloud included in the foreground object.
The device according to claim 50, wherein the types include: cars, bicycles, and pedestrians.
The device according to any one of claims 28-51, wherein the foreground object is a vehicle, and the device further comprises a post-processing module for:

Carry out statistics on traffic information in a specific area.
The device according to any one of claims 28-51, wherein the scene is a parking lot, the foreground object is a vehicle, and the device further comprises a post-processing module for:

Determine the parking space occupancy status of the parking lot according to the detected movement trajectory of the foreground object in the parking lot.
The device according to claim 53, wherein the post-processing module is further configured to:

Guide the vehicle to be parked to a free parking space.
A device for monitoring a moving object, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements claim 1 when the computer program is executed. To the steps of the method described in any one of 27.
A computer storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of claims 1 to 27 when the computer program is executed by a processor.