CN112639405A

CN112639405A - State information determination method, device, system, movable platform and storage medium

Info

Publication number: CN112639405A
Application number: CN202080004726.6A
Authority: CN
Inventors: 陆泽早
Original assignee: SZ DJI Technology Co Ltd
Current assignee: SZ DJI Technology Co Ltd
Priority date: 2020-05-07
Filing date: 2020-05-07
Publication date: 2021-04-09
Also published as: WO2021223166A1

Abstract

A method, apparatus, system, removable platform and storage medium for determining status information of a target object (102). The method for determining the state information of the target object (102) comprises the following steps: obtaining, by an imaging device carried by a movable platform, a plurality of frames of images about a target object (102); identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object (102); obtaining a plurality of ranging results between a movable platform and a target object (102); screening effective ranging results from the ranging results according to the information of the detection frames; and determining state information of the target object (102) according to the plurality of detection frame information and the effective ranging result.

Description

State information determination method, device, system, movable platform and storage medium

Technical Field

The present disclosure relates to the field of movable platform technologies, and in particular, to a method, an apparatus, a system, a movable platform, and a storage medium for determining state information of a target object.

Background

The movable platform includes an unmanned aerial vehicle ("UAV"), sometimes referred to as an "drone," which a user may remotely operate or program to achieve automatic flight, including unmanned aerial vehicles of various sizes and configurations. Of course, the movable platform is not limited thereto, and for example, the movable platform may also include a mobile device such as an unmanned vehicle, an unmanned ship, or the like.

The movable platform may be used for many purposes, such as for various personal, commercial, and tactical applications. The movable platform may be equipped with imaging means, such as a camera, video camera, etc. The movable platform equipped with the imaging device may determine status information of a target object, which may include, for example, a person, a moving object, a stationary object, and the like, which may include, for example, position information and the like.

BRIEF SUMMARY OF THE PRESENT DISCLOSURE

The present disclosure provides a method for determining state information of a target object, including: obtaining a plurality of frame images about a target object through an imaging device carried by a movable platform; identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object; obtaining a plurality of ranging results between the movable platform and the target object; screening effective ranging results from the ranging results according to the information of the detection frames; and determining the state information of the target object according to the plurality of detection frame information and the effective ranging result.

The present disclosure also provides another method for determining status information of a target object, including: obtaining a plurality of frame images about a target object through an imaging device carried by a movable platform; identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object; determining one or more target detection frame information satisfying a preset condition in the plurality of detection frame information; obtaining a plurality of ranging results between the movable platform and the target object; and determining the state information of the target object according to the one or more target detection frame information and the plurality of ranging results.

The present disclosure also provides a device for determining status information of a target object, including: a processor; a readable storage medium storing one or more programs, wherein the one or more programs, when executed by the processor, cause the processor to: obtaining a plurality of frame images about a target object obtained by an imaging device carried by a movable platform; identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object; obtaining a plurality of ranging results between the movable platform and the target object; screening effective ranging results from the ranging results according to the information of the detection frames; and determining the state information of the target object according to the plurality of detection frame information and the effective ranging result.

The present disclosure also provides another apparatus for determining status information of a target object, including: a processor; a readable storage medium storing one or more programs, wherein the one or more programs, when executed by the processor, cause the processor to: obtaining a plurality of frame images about a target object obtained by an imaging device carried by a movable platform; identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object; determining one or more target detection frame information satisfying a preset condition in the plurality of detection frame information; obtaining a plurality of ranging results between the movable platform and the target object; and determining the state information of the target object according to the one or more target detection frame information and the plurality of ranging results.

The present disclosure also provides a system for determining status information of a target object, including: an imaging device for obtaining a plurality of frame images about a target object; the status information determination apparatus as described above.

The present disclosure also provides a movable platform, including: a movable body; and a status information determination system as described above.

The present disclosure also provides a readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method as described above.

Through the embodiment of the disclosure, the invalid ranging result is identified according to the information of the plurality of detection frames, so that the effective ranging result is screened out from the plurality of ranging results, the state information of the target object is determined according to the information of the plurality of detection frames and the effective ranging result, the technical problem that the state estimation of the target object is invalid due to the fact that the state information of the target object is polluted by error data is at least partially solved, the false detection rate of the ranging result is reduced, and the reliability of the state information estimation is improved.

According to the embodiment of the disclosure, one or more pieces of target detection frame information meeting the preset condition in the plurality of pieces of detection frame information are determined, and the state information of the target object is determined according to the one or more pieces of target detection frame information and the plurality of ranging results, so that the technical problem that the state estimation of the target object is invalid due to the fact that the state information of the target object is polluted by error data is at least partially solved, and the reliability of the state information estimation is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:

fig. 1 schematically illustrates an application scenario in which a state information determination method of a target object may be applied according to an embodiment of the present disclosure.

Fig. 2 schematically illustrates a schematic diagram of one frame of image captured by an imaging device with respect to a target object according to an embodiment of the present disclosure.

Fig. 3 schematically shows a flowchart of a method of determining status information of a target object according to an embodiment of the present disclosure.

Fig. 4 schematically shows a flowchart for screening one or more ranging results corresponding to each detection frame information according to an embodiment of the present disclosure.

Fig. 5 schematically shows a flowchart for determining validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result according to an embodiment of the disclosure.

Fig. 6 schematically shows a flowchart for determining target detection frame information according to an embodiment of the present disclosure.

Fig. 7 schematically shows a flowchart for determining whether a target object moves according to a plurality of detection frame information, which satisfies a preset condition, according to an embodiment of the present disclosure.

Fig. 8 schematically illustrates a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to an embodiment of the present disclosure.

Fig. 9 schematically illustrates a schematic diagram of a probability distribution of initial position information corresponding to detection frame information according to another embodiment of the present disclosure.

Fig. 10 schematically illustrates a schematic diagram of determining a spatial location with a highest probability density from a superposition of a first probability distribution and a second probability distribution according to an embodiment of the disclosure.

Fig. 11 schematically illustrates a flowchart for determining status information of a target object according to each target detection frame information and a valid ranging result corresponding to each target detection frame information according to an embodiment of the present disclosure.

Fig. 12 schematically shows a time axis diagram for screening initial position information on a target object corresponding to each target detection frame information according to an embodiment of the present disclosure.

Fig. 13 schematically illustrates a schematic diagram of predicting location information of a target object according to an embodiment of the present disclosure.

Fig. 14 schematically shows a flowchart of a method of determining state information of a target object according to another embodiment of the present disclosure.

Fig. 15 schematically shows a block diagram of a state information determination system of a target object according to an embodiment of the present disclosure.

Detailed Description

The technical solution of the present disclosure will be clearly and completely described below with reference to the embodiments and the drawings in the embodiments. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

Some block diagrams and/or flow diagrams are shown in the figures. It will be understood that some blocks of the block diagrams and/or flowchart illustrations, or combinations thereof, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the instructions, which execute via the processor, create means for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks. The techniques of this disclosure may be implemented in hardware and/or software (including firmware, microcode, etc.). In addition, the techniques of this disclosure may take the form of a computer program product on a computer-readable storage medium having instructions stored thereon for use by or in connection with an instruction execution system.

The movable platform may be configured as an imaging device, allowing a user to remotely determine location information of a target object, which may include, for example, a person, a moving object, a stationary object, etc., and to track or photograph the target object, among other applications. Taking the example of tracking a target object, this ability to track the target object allows the movable platform to autonomously operate other devices such as an imaging device while tracking or filming a moving target object to facilitate imaging of the target object.

As a schematic illustration, a movable platform (e.g., a UAV) may be configured to autonomously track the movement of an object and adjust the speed and direction of its movement accordingly, while adjusting the orientation of the imaging device to maintain a predetermined relative position from the object. With this configuration, the UAV may maintain a predetermined field of view for the object such that images of the object may be captured with substantially the same range and accuracy as the object moves.

In practical application, a machine learning algorithm can be used to identify a target object to be tracked on an image acquired by an imaging device, so as to obtain a detection frame of the target object in the image, determine the position of the target object according to the detection frame of the target object, and change the position of the movable platform, the posture of the imaging device and the like according to the position of the target object, thereby tracking the target object.

In the process of implementing the present disclosure, it is found that the position of the target object is determined according to the detection frame of the target object, and the accuracy and reliability are not high, for example, when the target object is not located at the center of the acquired image, the distance between the movable platform and the target object cannot be obtained more accurately, so that the position of the target object cannot be determined more accurately.

However, when the position of the target object is determined, the detection frame of the target object can be combined with the ranging result of the target object, and the accuracy and reliability of the state information of the target object can be improved. Therefore, how to obtain the state information of the target object, such as the position and the track, by combining the detection frame of the target object and the ranging result is a problem to be solved urgently at present.

The position and distance information of the target object can be provided by combining a visual detection frame of the target object with a plurality of ranging methods, which are commonly known as: satellite navigation systems, lidar, ToF (Time of Flight Time ranging, short for ToF) ranging, binocular ranging, triangulation based on parallax, and ranging based on other a priori knowledge (e.g., the ground altitude of the target object, the height of the target object itself).

These methods are both good and bad, and provide a challenge for the process of fusing multiple modes to determine the position of the target object. For example, the positioning accuracy of the satellite navigation system is low, but the satellite navigation system is not influenced by the occlusion of visual disturbance; the laser radar and the ToF have high ranging precision and large ranging range, but no barrier is required between the UAV and the target object; the binocular range finding precision is limited by the size of the UAV, and the precision of a long-distance target object is poor; performing triangular ranging according to parallax, wherein the method is only suitable for a static target object, a UAV is required to fly in a fixed track, and an imaging device shoots the target object at two different visual angles; the accuracy of ranging according to the prior knowledge depends on the degree of conformity of the prior assumption with the actual situation. Therefore, an improved method for determining the state information of the target object is needed to enhance the adaptability and reliability of determining the state information of the target object.

According to the embodiment of the present disclosure, an improved method for determining status information of a target object is provided, which may identify a detection frame of the target object through a machine learning method, measure a distance (e.g., a single-point laser radar) of the target object relative to a movable platform through a ranging device, and combine the two data to obtain real-time or near real-time status information of the target object, where the status information of the target object may include, for example, a current position, a speed, a historical track, and the like.

An application scenario in which the method for determining the state information of the target object provided by the embodiment of the present disclosure may be applied will be described below by taking the movable platform as the unmanned aerial vehicle UAV as an example.

Fig. 1 schematically illustrates an application scenario in which a state information determination method of a target object may be applied according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a scenario in which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.

As shown in fig. 1, the field of view of the imaging device of the drone 101 may include, but is not limited to, a target object 102, an object 103, an object 104, and the like. Illustratively, the drone 101 may capture one or more frames of images about the target object 102 by the imaging device, and by identifying each frame of image, detection frame information about the target object 102 may be obtained. According to an embodiment of the present disclosure, the target object 102 may be in a moving state or may be in a stationary state. In a case where the target object 102 is in a certain state, particularly a moving state, the imaging apparatus may capture an image about the target object 102 in real time or at preset time intervals, and then recognize the captured image, resulting in a plurality of detection frame information.

As shown in fig. 2, an image 200 captured by the imaging device includes a target object 102, an object 103, and an object 104. The image 200 may be recognized using a machine learning based neural network, resulting in one detection frame information 1021 for the target object 102.

The drone 101 may obtain the distance of the target object 102 relative to the drone 101 through ranging device measurements (e.g., single point lidar). In a certain state, especially a moving state, of the target object 102, the ranging device may measure the distance of the target object 102 relative to the drone 101 in real time or at preset time intervals, so as to obtain a plurality of ranging results.

By combining the detection frame information and the ranging result of the target object 102, real-time or near real-time status information of the target object 102 can be obtained, for example, position information, speed information, and historical track of the target object, etc. can be obtained.

Embodiments of the present disclosure may be applicable to a variety of applications based on state information of a target object. For example, the target state information obtained by the embodiment of the present disclosure can be further used in the fields of target object tracking, target object surrounding, imaging device orientation control, imaging device tracking focusing, tracking zooming, target object position prediction, feedback optimization visual target recognition, AR (Augmented Reality, abbreviated as AR) monitoring, and the like, and realizes scene applications such as controlling the movable platform and the imaging device carried by the movable platform to track the target object.

It should be noted that, unless explicitly stated that there is an execution sequence between different operations or there is an execution sequence between different operations in technical implementation, the execution sequence between multiple operations may not be sequential, or multiple operations may be executed simultaneously in the flowchart in this disclosure.

For example, in fig. 3, there is a precedence order between operation S301 and operation S302 in technical implementation, and there is no precedence order between operation S303 and operation S301 and operation S302 in technical implementation, and it is not explicitly stated that operation S301 and operation S302 are performed before operation S303. Therefore, although operations S301 and S302 precede operation S303 in the flowchart shown in fig. 3, the operations S301 and S302 may be performed before or simultaneously with operations S301 and S302 when operation S303 is performed.

As shown in fig. 3, the method of determining the state information of the target object includes operations S301 to S305.

In operation S301, a plurality of frames of images about a target object are obtained by an imaging device carried by a movable platform.

In operation S302, each of the plurality of frame images is identified, and a plurality of detection frame information about the target object is obtained.

According to embodiments of the present disclosure, an imaging device may continuously capture images over its field of view. The captured image is subjected to image recognition to obtain a multi-frame image including the target object, but the target object may not be included in the image captured by the imaging device.

According to the embodiment of the present disclosure, for example, each captured frame of image may be recognized by a machine learning method, and if a target object is included in the image, one detection frame information about the target object may be obtained. If the image has no target object, the image can be marked as an invalid image, and further, the invalid image can be filtered out, or if the image has no target object, the image can be directly filtered out.

According to an embodiment of the present disclosure, the detection frame information may include various information, for example, may include part or all of the following various information: the position and size of the target object on the image screen, the angle of view, the position and orientation of the imaging device, the sampling time point of the image, and the like.

According to an embodiment of the present disclosure, the detection frame (as shown in fig. 2) may be represented by a rectangular frame, and the position and size of the target object on the image screen may be provided by the target recognition module. The field angle of the imaging device may be provided by a zoom module of the imaging device, which may perform the following algorithms (one) and (two) to obtain the field angle of the imaging device.

Wherein, fov_zxAngle of view indicating the direction of the lines of the image screen, fov_zyThe field angle indicating the column direction of the image screen, focal-length is the focal length of the imaging device in real time, and W, H are the width and height of the image sensor of the imaging device.

According to embodiments of the present disclosure, the pose of the imaging device relative to the world coordinate system, and the corresponding sampling time, may be provided by the vehicle pose measurement module. According to embodiments of the present disclosure, the position of the imaging device in the world coordinate system, and the corresponding sampling time, may be provided by the vehicle position measurement module.

In operation S303, a plurality of ranging results between the movable platform and the target object are obtained.

According to the embodiment of the disclosure, a plurality of ranging results between the movable platform and the target object can be measured by the ranging device. The type of the distance measuring device is not limited.

For example, the distance of the target object relative to the drone may be obtained by a single point lidar. Wherein the single point lidar may be arranged on a moveable platform. Of course, the present disclosure is not limited to obtaining ranging results by a single-point lidar on the movable platform, and multiple ranging results between the movable platform and the target object may also be obtained by other ranging devices such as ToF camera, single-line lidar, area-array lidar, binocular camera, etc.

According to the embodiment of the disclosure, the ranging device may measure each ranging result according to a time sequence, and each ranging result has a corresponding sampling time point.

According to the embodiment of the present disclosure, the ranging apparatus may use a single-point laser radar having characteristics of a long ranging distance and a high precision, and the ranging result between the movable platform and the target object may be a laser ranging result. The imaging device can use a monocular camera, and the cost is lower than that of the laser radar and the camera array. The real-time distance between the movable platform and the target object is measured by adopting the single-point laser radar, a satellite navigation system is not required to provide the space position of the target object, and the relative height between the imaging device and the target object is not changed or is basically unchanged. By using a low-cost monocular camera and a single-point laser radar, the continuous ranging of the long-distance target object is reliably realized, and the adaptability and the reliability of the position estimation of the target object are improved. Through the embodiment of the disclosure, invalid laser measurement results can be filtered, and the false detection rate of laser is reduced.

In operation S304, a valid ranging result is screened from the plurality of ranging results according to the plurality of detection frame information.

According to the embodiments of the present disclosure, the sampling frequency of the imaging device and the sampling frequency of the distance measuring device may be the same or different. Each detection frame information may have a corresponding one ranging result, or each detection frame information may have a corresponding plurality of ranging results.

When effective ranging results are screened out from a plurality of ranging results according to a plurality of detection frame information, one or a plurality of ranging results corresponding to each detection frame information can be determined; and then screening one or more ranging results corresponding to each detection frame information.

According to the embodiment of the present disclosure, each detection frame information has a corresponding sampling time point, and the sampling time point of the image corresponding to the detection frame information may be used as the sampling time point corresponding to the detection frame information.

When determining one or more ranging results corresponding to each detection frame information, the one or more ranging results corresponding to each detection frame information may be determined according to a sampling time point of each detection frame information and a sampling time point of each ranging result in the obtained plurality of ranging results.

According to an embodiment of the present disclosure, each ranging result may be associated to the detection frame information closest to the sampling time point of the ranging result. Taking the example of associating the ranging result with one of the two detection frame information, for a certain ranging result, subtracting the sampling time point of the first detection frame information from the sampling time point corresponding to the ranging result to obtain a first time difference, subtracting the sampling time point of the second detection frame information from the sampling time point corresponding to the ranging result to obtain a second time difference, comparing the absolute values of the first time difference and the second time difference, and determining the corresponding detection frame information with the smaller absolute value as the detection frame information closest to the sampling time point of the ranging result, thereby determining one or more ranging results corresponding to each detection frame information.

According to the embodiment of the disclosure, an invalid ranging result can be identified according to a plurality of detection frame information, so that a valid ranging result can be screened out from a plurality of ranging results.

In operation S305, status information of the target object is determined according to the plurality of detection frame information and the valid ranging result.

According to an embodiment of the present disclosure, the valid ranging result may include one or more. The effective ranging result corresponding to each detection frame information can be determined firstly; and then determining the state information of the target object according to each detection frame information and the effective ranging result corresponding to each detection frame information.

According to the embodiments of the present disclosure, for example, the state information of the target object may be determined according to the angle of view, the position and the posture of the imaging device corresponding to each detection frame and the corresponding effective ranging result.

According to the embodiment of the disclosure, the effective ranging result corresponding to each detection frame information can be determined according to the sampling time point.

For example, each valid ranging result may be associated to the detection frame information closest to the sampling time point of the valid ranging result according to the sampling time point of each detection frame information and the sampling time point of each valid ranging result. For a specific method, reference may be made to the above example of associating the ranging result with one of the two detection frame information, which is not described herein again.

According to an embodiment of the present disclosure, in a case that each detection frame information corresponds to a plurality of valid ranging results, determining state information of a target object according to each detection frame information and the valid ranging result corresponding to each detection frame information includes: calculating a weighted average value of a plurality of effective ranging results corresponding to each detection frame information to obtain a target ranging result corresponding to each detection frame information; and determining the state information of the target object according to each detection frame information and the target ranging result corresponding to each detection frame information.

According to the embodiment of the disclosure, a result obtained by dividing the echo intensity of each effective ranging result corresponding to the detection frame information by a reference value can be used as the weight of the effective ranging result, wherein the reference value is the sum of the echo intensities of all effective ranging results corresponding to the detection frame information; and then calculating to obtain a weighted average value according to the plurality of effective ranging results and the weight of each effective ranging result.

According to an embodiment of the present disclosure, the reference value may be a sum of echo intensities of a plurality of valid ranging results corresponding to the detection frame information.

For example, for the first detection frame information, the effective ranging results corresponding to the first detection frame information include effective ranging results 1 to 4, and the echo intensities corresponding to the effective ranging results 1 to 4 are echo intensities 1 to 4. The weight 1 of the effective ranging result 1 is echo intensity 1/(echo intensity 1+ echo intensity 2+ echo intensity 3+ echo intensity 4), the weight 2 of the effective ranging result 2 is echo intensity 2/(echo intensity 1+ echo intensity 2+ echo intensity 3+ echo intensity 4), the weight 3 of the effective ranging result 3 is echo intensity 3/(echo intensity 1+ echo intensity 2+ echo intensity 3+ echo intensity 4), and the weight 4 of the effective ranging result 4 is echo intensity 4/(echo intensity 1+ echo intensity 2+ echo intensity 3+ echo intensity 4).

Then, the target ranging result corresponding to the first detection frame information is equal to valid ranging result 1 × weight 1+ valid ranging result 2 × weight 2+ valid ranging result 3 × weight 3+ valid ranging result 4 × weight 4.

According to an embodiment of the present disclosure, the state information of the target object includes, but is not limited to, position information, velocity information, and a history trajectory of the target object in the world coordinate system. The historical track comprises a plurality of track points.

Meanwhile, the state information of the target object is determined by combining the detection frame information and the effective ranging result, so that the acquired image data has no high requirement, a depth sensor can be omitted for acquiring a depth image, and the problems of small effective range or low precision of the acquired depth image caused by the limitation of the depth sensor are at least partially solved.

The method shown in fig. 3 is further described with reference to fig. 4-13 in conjunction with specific embodiments.

As shown in fig. 4, the screening of the one or more ranging results corresponding to each detection frame information may include operations S401 to S403.

In operation S401, a laser spot corresponding to each of one or more ranging results corresponding to each detection frame information is determined.

According to an embodiment of the present disclosure, taking the ranging result as a laser ranging result as an example, the ranging result lidar may be respectively calculated using a field angle of the imaging device and a scattering angle of the laser radar, for example_m...lidar_n-1、lidar_nCorresponding laser spot circle in picture_m...circle_n-1、circle_n。

In operation S402, validity of each ranging result is determined according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result.

According to the embodiment of the disclosure, the area coincidence rate of the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result can be determined, and then the validity of each ranging result is determined according to the area coincidence rate.

According to the embodiment of the disclosure, each ranging result has corresponding detection frame information and a laser spot. The area of the detection frame can be calculated according to the detection frame information (such as the length and the width of the detection frame in the picture) corresponding to the ranging result, the area of the laser spot is divided by the area of the detection frame to obtain the area coincidence rate between the detection frame information corresponding to the ranging result and the laser spot, and then the effectiveness of each ranging result is determined according to the area coincidence rate.

According to an embodiment of the present disclosure, the area coincidence rate may be compared with a preset proportion threshold; determining the ranging result with the area coincidence rate larger than or equal to a preset proportion threshold value as an effective ranging result; and determining the ranging result with the area coincidence rate smaller than the preset proportional threshold as an invalid ranging result.

According to an embodiment of the present disclosure, the size of the preset proportion threshold may be preset empirically. Illustratively, for example, the size of the preset proportion threshold may be 70%. If the area coincidence rate is larger than or equal to 70%, the ranging result can be marked as valid, otherwise, the ranging result is marked as invalid, and then the invalid ranging result is filtered according to the marking result.

In operation S403, one or more ranging results corresponding to each detection frame information are filtered according to the validity of each ranging result.

According to the embodiment of the disclosure, the invalid ranging result may be filtered from one or more ranging results corresponding to the detection frame information, and the valid ranging result may be determined as the ranging result corresponding to the detection frame information.

As shown in fig. 5, determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result may include operations S501 to S502.

In operation S501, interpolation detection frame information corresponding to each ranging result in a plurality of ranging results between sampling times of two pieces of detection frame information adjacent to each sampling time is determined according to the two pieces of detection frame information adjacent to each sampling time, and detection frame information corresponding to each ranging result is obtained.

According to an embodiment of the present disclosure, for example, for any two detection frame information boxes including a target object whose sampling times are adjacent_aAnd box_bThe corresponding sampling time points are respectively t_aAnd t_bAnd select t_aAnd t_bAll ranging results lidar in the time period in between_m...lidar_n-1、lidar_nThe sampling time points corresponding to the ranging results are respectively t_m、t_m+1...t_n-1、t_nWherein t is satisfied_a＜t_m...t_n-1、t_n≤t_b。

Suppose that during this time period t_aTo t_bIn, the detection frame is from box_aUniform speed change to box_bUsing an interpolation algorithm to obtain t_m...t_n-1、t_nRespectively corresponding interpolation detection frames box_m...box_n-1、box_n。

Respectively calculating the distance measurement results lidar_m……lidar_n-1、lidar_nCorresponding laser spot circle in picture_m……circle_n-1、circle_n. For example, the laser spot may be calculated using the angle of view of the imaging device and the scattering angle of the laser radar.

In operation S502, validity of each ranging result is determined according to a laser spot corresponding to the ranging result and detection frame information corresponding to the ranging result.

According to the embodiment of the disclosure, the validity of each ranging result can be determined according to the area coincidence rate of the detection frame information corresponding to the ranging result and the laser spot corresponding to the ranging result. For a specific method, reference may be made to the description of fig. 4, which is not described herein again.

According to the embodiment of the disclosure, in particular, for each newly generated detection frame, a previous detection frame with a sampling time closest to the newly generated detection frame may be determined, and the distance measurement result between the newly generated detection frame and the closest previous detection frame with the sampling time may be filtered. The method and the device can realize real-time or near real-time screening of the ranging result, thereby achieving the effect of more accurately determining the state information of the target object.

According to the embodiment of the disclosure, the detection frame information of the target object is obtained through a machine learning method, the ranging results of the target object relative to the movable platform are obtained through the ranging device, the ranging results are screened through the detection frame information to obtain effective ranging results, and the real-time or near-real-time state information of the target object can be obtained by combining the detection frame information and the effective ranging results of the target object.

According to the embodiment of the disclosure, after the effective ranging result corresponding to each detection frame information is determined, the physical estimated size of the target object corresponding to each detection frame information can be further determined; and screening the effective ranging result corresponding to each detection frame information according to the physical estimation size of the target object corresponding to each detection frame information.

According to the embodiment of the disclosure, the field angle corresponding to each piece of detection frame information can be determined according to each piece of detection frame information and the field angle of the imaging device when each piece of detection frame information is collected; and then determining the physical estimation size corresponding to each detection frame information according to the field angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.

For one detection frame information, the field angle of the detection frame is first calculated according to the field angle of the imaging device and the size of the detection frame, and then, the physical estimated size corresponding to the detection frame information, which is the estimated size of the target object in the real world, is calculated according to the effective ranging result and the field angle of the detection frame.

According to an embodiment of the present disclosure, specifically, calculating the physical estimated size of the target object corresponding to the detection frame information may include the following operations.

First, according to the real-time field angle of the image frame and the position and size of the target object on the image frame, the corresponding course angles of the left edge and the right edge of the rectangular frame of the target object on the image frame relative to the imaging device are calculated (i.e. the included angle formed by the connecting lines from the optical center of the imaging device to the upper, lower, left and right four corners of the rectangular frame can be determined by using the pixel offset and the real-time field angle of the image frame), and the corresponding pitch angles of the upper edge and the lower edge of the rectangular frame of the target object on the image frame relative to the imaging device are calculated.

And subtracting the course angles respectively corresponding to the left edge and the right edge to obtain a course angle range of the target relative to the imaging device, subtracting the pitch angles respectively corresponding to the upper edge and the lower edge to obtain a pitch angle range of the target relative to the imaging device, wherein the field angle of the target object relative to the course angle range and the pitch angle range of the imaging device is simply referred to as the field angle of the target detection frame below.

Then, after the field angle of the detection frame is determined, the arc lengths corresponding to the heading angle and the pitch angle can be determined by using the field angle of the detection frame and the effective distance measurement result, and the range of four sides of the rectangular frame can be determined by using the end points of the two arc lengths, so that the physical estimation size of the target detection frame can be determined.

The reasonable range of the physical size of the target object can be estimated according to the prior knowledge of the type of the target object, and unreasonable effective ranging results are filtered.

According to an embodiment of the present disclosure, the screening the effective ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information may include: comparing the physical estimation size of the target object corresponding to each detection frame information with a preset reasonable range; and filtering the effective ranging result corresponding to the detection frame information under the condition that the physical estimation size of the target object corresponding to the detection frame information does not accord with the preset reasonable range.

According to the embodiment of the disclosure, the object type of the target object can be determined first, wherein each object type has a corresponding preset reasonable range; then determining a target preset reasonable range according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each piece of detection frame information with a target preset reasonable range.

For example, if the target object is a human, the target preset reasonable range may be 0.6 meters to 2 meters, and the calculated physical estimated size may be compared with 0.6 meters to 2 meters. If the target object is a vehicle, the target preset reasonable range may be 2 meters to 20 meters, and the calculated physical estimated size may be compared with 2 meters to 20 meters.

According to the embodiment of the disclosure, after filtering the effective ranging result corresponding to the detection frame information, if the detection frame information does not have the corresponding effective ranging result, the detection frame information can be filtered.

Of course, according to the embodiment of the present disclosure, the detection information may not be filtered, and after filtering the effective ranging result corresponding to the detection frame information, if the detection frame information does not have a corresponding effective ranging result, the effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information from which the effective ranging result is filtered may be determined as the effective ranging result corresponding to the detection frame information from which the effective ranging result is filtered.

According to an embodiment of the present disclosure, when determining the state information of the target object according to the plurality of detection frame information and the effective ranging result, one or more pieces of target detection frame information satisfying a preset condition among the plurality of detection frame information may be determined first, and the state information of the target object may be determined according to each piece of target detection frame information and the effective ranging result corresponding to each piece of target detection frame information, and may not be used for other detection frames except the target detection frame information.

According to the embodiment of the disclosure, the valid ranging result corresponding to each target detection frame information in the one or more target detection frame information may be determined first, in other words, the target detection frame information is associated with the corresponding valid ranging result first, and then the state information of the target object is determined according to each target detection frame information and the valid ranging result corresponding to each target detection frame information.

As shown in fig. 6, determining the target detection frame information may include operations S601 to S602.

In operation S601, it is determined whether a target object has moved to satisfy a preset condition according to a plurality of detection frame information.

According to the embodiment of the disclosure, the imaging device may capture a plurality of frames of images in a time period, recognize the plurality of frames of images to obtain a plurality of detection frame information, and determine whether the target object has a large movement in the time period according to the plurality of detection frame information to determine whether the target object has a movement satisfying a preset condition.

In operation S602, in the case where the target object has moved while satisfying the preset condition, the detection frame information corresponding to the movement while the target object satisfies the preset condition is determined as the target detection frame information.

According to the embodiments of the present disclosure, for example, if the target object has a large movement within a time period, the corresponding detection frame information of the target object when the target object has the large movement may be determined as the target detection frame information. It should be noted that, when the target object moves greatly, the corresponding detection frame information may include a plurality of pieces, and therefore, a plurality of pieces of target detection frame information may be obtained. Further, a plurality of target detection frame information may be marked for use in determining status information of the target object.

According to an embodiment of the present disclosure, whether or not a target object has moved satisfying a preset condition will be described next.

As shown in fig. 7, determining whether the target object has moved to satisfy the preset condition according to the plurality of detection frame information may include operations S701 to S704.

In operation S701, first state information of the imaging apparatus at the time of acquisition of the first detection frame information and second state information of the imaging apparatus at the time of acquisition of the second detection frame information are obtained for any adjacent first detection frame information and second detection frame information among the plurality of detection frame information.

According to the embodiment of the present disclosure, for example, a plurality of pieces of detection frame information within one time period may be acquired, and for first detection frame information and second detection frame information in which any two sampling time points within the one time period are adjacent, state information of the imaging apparatus at the time of acquisition of each piece of detection frame information is acquired. The state information of the imaging device may include, among others, an orientation of the imaging device, a field angle of the imaging device, and a position of the imaging device.

In operation S702, a first probability distribution of initial position information on the target object corresponding to the first detection frame information is determined according to the effective ranging result corresponding to the first detection frame information and the first state information.

In operation S703, a second probability distribution of initial position information on the target object corresponding to the second detection frame information is determined according to the effective ranging result corresponding to the second detection frame information and the second state information.

According to the embodiment of the disclosure, for example, any two detection boxes in a time period are selected_aAnd box_bAnd search and detect the box_aAnd box_bIs sampled at time t_aAnd t_bThe orientation of the corresponding imaging device, the field angle of the imaging device, the position of the imaging device (hereinafter referred to as detection frame information), and the effective ranging result. Then, for each detection frame, calculating the probability score of the initial position information corresponding to the detection frame information according to the detection frame information and the effective ranging result by using a back projection algorithmThe probability distribution characterizes the probability that the detection box maps to various spatial positions in the physical spatial domain.

When the detection box 801 has a corresponding valid ranging result, the probability distribution of the initial position information is a flowerpot type (or referred to as a circular truncated cone type). As shown in fig. 8, the position of the target object may be used as an original position, a part before the effective distance measurement result which is 0.95 times as long as the target object and a part after the effective distance measurement result which is 1.05 times as long as the target object are cut off, and the probability distribution of the target object is obtained as a flowerpot shape, the effective distance measurement result may be a distance between the imaging device and the target object, the upper and lower limits of the effective distance measurement result are positively correlated with the measurement error of the measurement device, and the closer to the center, the higher the probability.

When the detection frame 901 has no corresponding valid ranging result (for example, the ranging result corresponding to the detection frame information is determined to be invalid according to the physical estimated size of the target object corresponding to the detection frame information), the probability distribution of the initial position information is a conical type, and as shown in fig. 9, the closer to the conical axis, the higher the probability. According to the embodiment of the present disclosure, the effective ranging result corresponding to the detection frame information adjacent to the sampling time point of the detection frame 901 may be determined as the effective ranging result corresponding to the detection frame 901. Then, the conical probability distribution is cut into the flowerpot type probability distribution by the method described in fig. 8 according to the effective ranging result corresponding to the detection frame 901.

In operation S704, it is determined whether the target object has moved according to the first probability distribution and the second probability distribution, which satisfy a preset condition.

According to an embodiment of the present disclosure, determining whether the target object has moved satisfying a preset condition according to the first probability distribution and the second probability distribution includes the following operations.

First, the spatial location with the highest probability density is determined from the first probability distribution and the second probability distribution. For example, the first probability distribution and the second probability distribution are superimposed, and the spatial position with the highest probability density is calculated. It should be noted that, for the detection frame information without the corresponding valid laser ranging result, the spatial position with the highest probability density may also be determined by means of probability distribution superposition.

As shown in fig. 10, first, a first probability distribution corresponding to the first detection frame information 1001 and a second probability distribution corresponding to the second detection frame information 1002 are superimposed, and a spatial position 1003 having the highest probability density can be specified.

It should be noted that, no matter whether the probability distributions corresponding to the two detection frame information overlap or not, the spatial position with the highest probability density can be determined.

Then, a first distance from a spatial position with the highest probability density to a center position of a first probability distribution of the first probability distribution and a second distance from a spatial position with the highest probability density to a center position of a second probability distribution of the second probability distribution are calculated, respectively. For example, a mahalanobis distance (euclidean distance multiplied by the probability distribution coefficient) from the center position of the probability distribution can be calculated for each of the two probability distributions for the spatial position having the highest probability density.

Next, a probabilistic distance between the first detection frame information and the second detection frame information may be determined according to the first distance and the second distance. For example, the sum of the first distance and the second distance is taken as the probability distance between the two detection frames.

And finally, comparing the probability distance between the first detection frame information and the second detection frame information with a preset threshold, and if the probability distance is greater than or equal to the preset threshold, determining that the target object moves to meet the preset condition. The magnitude of the preset threshold may reflect the amplitude requirement of the target object movement, for example, the larger the preset threshold, the higher the amplitude requirement of the target object movement. The preset threshold may be preset according to the actual effect, and optionally, the preset threshold may be 1.0.

According to the embodiment of the disclosure, if the probability distance between any two detection frames in a time period is greater than a preset threshold, it can be considered that the target object has a large motion in the time period. The length of the time period may be preset, and may be 2 seconds, 5 seconds, or the like, for example.

According to the embodiment of the disclosure, the state information of the target object can be determined according to each piece of target detection frame information and the effective ranging result corresponding to each piece of target detection frame information. For example, initial position information of the target object may be determined according to the orientation, the field angle, the position, and the effective ranging result of the imaging device at the time of the target detection frame acquisition. The target detection frame information may be detection frame information in which a large motion occurs.

As shown in fig. 11, determining the state information of the target object according to each target detection frame information and the valid ranging result corresponding to each target detection frame information may include operations S1101 to S1103.

In operation S1101, initial position information about the target object corresponding to each target detection frame information is determined according to each target detection frame information and a valid ranging result corresponding to each target detection frame information, resulting in a plurality of initial position information.

In operation S1102, a plurality of initial position information is screened according to a sampling time point corresponding to each target detection frame information, so as to obtain one or more effective initial position information.

According to the embodiment of the disclosure, the initial position information about the target object corresponding to each piece of target detection frame information may be sequentially filtered according to the time sequence of the sampling time points.

In operation S1103, status information of the target object is determined according to the one or more valid initial position information.

According to an embodiment of the present disclosure, for example, a motion trajectory of a target object may be generated from a plurality of valid initial position information. Alternatively, the moving speed, acceleration, etc. of the target object may be calculated from a plurality of effective initial position information,

according to an embodiment of the present disclosure, when determining the state information of the target object according to the one or more effective initial position information, the one or more effective initial position information may be optimized to smooth the motion trajectory with respect to the target object. And obtaining final optimized position information of the target object according to the finally optimized motion trail.

According to the embodiment of the disclosure, sequentially screening the initial position information about the target object corresponding to each piece of target detection frame information in time sequence includes the following operations.

Firstly, calculating the time difference between the initial position information currently being screened and the sampling time point of the next initial position information adjacent to the sampling time of the initial position information currently being screened; then comparing the time difference with a state variable threshold; if the time difference is smaller than the state variable threshold, the initial position information currently being screened can be filtered; if the time difference is greater than or equal to the state variable threshold, the initial position information currently being screened may be retained, wherein the retained initial position information currently being screened is valid initial position information.

According to the embodiments of the present disclosure, for example, from the detection frames within a past period of time, the target detection frame box in which the target object is largely moved is selected_m…box_n-1、box_nAnd sampling time t of the detection frames_m…t_n-1、t_nCorresponding initial position information pos_m…pos_n-1、pos_nAs the state variable to be screened.

Then, according to the target detection box_m…box_n-1、box_nCorresponding associated data (e.g. imaging device)Orientation of the position, angle of view of the imaging device, position of the imaging device, and effective ranging result) using a back projection algorithm, initial position information pos is calculated_m…pos_n-1、pos_nIf the target detection frame has no corresponding effective ranging result, the previous effective ranging result adjacent to the sampling time can be adopted for calculation to obtain the initial value.

And finally, calculating the time difference between each state variable to be screened and the next state variable to be screened, and removing the state variable of which the time difference with the next state variable to be screened is smaller than the threshold value of the state variable from all the state variables to be screened. The latter state variable to be screened is the latter initial position information adjacent to the sampling time of the initial position information currently being screened.

According to an embodiment of the present disclosure, the state variable threshold may be a fixed value that is set in advance, for example, empirically.

According to embodiments of the present disclosure, the state variable threshold may also vary with the determined valid initial position information.

According to an embodiment of the present disclosure, the state variable threshold is a time difference between the initial position information currently being screened and a sampling time point of valid initial position information adjacent to a sampling time of the initial position information currently being screened.

The following describes a specific example of sequentially filtering the initial position information about the target object corresponding to each piece of target detection frame information in time order.

For example, a target detection frame box in which a target object is greatly moved is selected₁～box₆Corresponding to a sampling time point of t₁～t₆。

As shown in FIG. 12, t₁～t₆Sequentially increasing according to time sequence, sampling time t₆Corresponding target detection frame box₆May be the last current test box. By sampling the above method, initial position information pos can be calculated₁～pos₆Initial value of (2), initial position information pos₁～pos₆The initial value of (1) corresponds to the state variable to be screened (1) to the state variable to be screened (6), and the state variable to be screened (6) is taken as the last state variable to be screened. According to the embodiment of the disclosure, for the state variable 6 to be screened, as the current latest detection frame is adopted, the state variable can be directly used as an effective state variable without being eliminated.

In the process of screening along with the initial position information of the target object, all state variables to be screened may be processed according to a reverse time sequence, as shown in fig. 12, that is, a state variable 5 to be screened is processed first, and then a state variable 4 to be screened to a state variable 1 to be screened are processed in sequence. The state variable 6 to be screened is a state variable to be screened next to the state variable 5 to be screened, the state variable 5 to be screened is a state variable to be screened next to the state variable 4 to be screened, the state variable 4 to be screened is a state variable to be screened next to the state variable 3 to be screened, and so on.

For the state variable 5 to be screened, the time difference t between it and the state variable 6 to be screened is calculated₆-t₅Will t₆-t₅Comparing with a fifth preset threshold value if t₆-t₅If the initial position information pos is smaller than a fifth preset threshold value, filtering the initial position information pos₅(ii) a If t is₆-t₅If the initial position information pos is larger than or equal to a fifth preset threshold value, the initial position information pos is reserved₅Retained initial position information pos₅As valid state variables.

For the state variable 4 to be screened, the time difference t between the state variable and the state variable 5 to be screened is calculated₅-t₄Will t₅-t₄Comparing with a fourth preset threshold value if t₅-t₄If the initial position information pos is smaller than a fourth preset threshold value, filtering the initial position information pos₄(ii) a If t is₅-t₄If the initial position information pos is larger than or equal to a fourth preset threshold value, the initial position information pos is reserved₄Retained initial position information pos₄As valid state variables.

And by analogy, for the remaining state variables to be screened, calculating the time difference between the state variables to be screened and the next state variable to be screened, and comparing the time difference with the corresponding preset threshold value.

For example, for the state variable 1 to be screened, the time difference t between it and the state variable 2 to be screened is calculated₂-t₁Will t₂-t₁Comparing with a first preset threshold value if t₂-t₁If the initial position information pos is smaller than a first preset threshold value, filtering the initial position information pos₁(ii) a If t is₂-t₁If the initial position information pos is larger than or equal to a first preset threshold value, the initial position information pos is reserved₁Retained initial position information pos₁As valid state variables.

Further, the first to third preset thresholds may be varied with the determined valid state variable. For example, the state variable threshold may be a time difference between the state variable currently being screened and a sampling time point of the valid state variable adjacent to the sampling time of the state variable currently being screened.

All the state variables to be screened are processed in reverse time order, specifically, for example, the current state variable to be screened is the state variable 5 to be screened, and the next valid state variable is the state variable 6 to be screened, then the fifth preset threshold is equal to t₆-t₅Time difference t between state variable 5 to be screened and state variable 6 to be screened₆-t₅Is equal to a fifth preset threshold t₆-t₅Therefore, the state variable 5 to be screened is a valid state variable.

For the state variable 4 to be screened, the next valid state variable is the state variable 5 to be screened, then, the fourth preset threshold is equal to t₅-t₄If the time difference t between the state variable 4 to be screened and the state variable 3 to be screened is present₄-t₃Greater than t₅-t₄If the state variable 4 to be screened is an effective state variable, otherwise, the state variable 4 to be screened is an ineffective state variable, and the state variable is rejected. If the state variable 4 to be screened is presentThe effective state variable, at this time, the next effective state variable is the state variable 4 to be screened for the state variable 3 to be screened, and then, the third preset threshold is equal to t₄-t₃If the state variable 4 to be screened is an invalid state variable, and at this time, the next valid state variable is the state variable 5 to be screened for the state variable 3 to be screened, then the third preset threshold is equal to t₅-t₃。

Therefore, the third preset threshold value varies with whether the state variable 4 to be screened is a valid state variable, in other words, the third preset threshold value varies with the determined valid state variable. Similarly, the second preset threshold and the first preset threshold are changed along with the determined valid state variable, and are not described herein again. Wherein, the valid state variable is the valid initial position information.

According to an embodiment of the present disclosure, after obtaining the one or more valid initial position information, the one or more valid initial position information may be optimized. For example, one or more valid initial position information may be non-linearly optimized to minimize a target offset, where the target offset is associated with the detection frame information and/or valid ranging results, and each valid initial position information is non-linearly optimized to have corresponding optimized position information.

According to an embodiment of the present disclosure, the target deviation may include a first deviation and/or a second deviation; the first deviation comprises an observation deviation between the effective initial position information and target detection frame information and an effective ranging result which are used for calculating the effective initial position information; the second deviation comprises a deviation of the degree of smoothness between adjacent valid initial position information from the prior value.

According to an embodiment of the present disclosure, for example, the target deviation may be the following algorithm (three):

minimizing the target deviation may be

Wherein alpha is_i(x) Characterizing a first deviation, β_j(x_j，x_j+1，x_j+2) And characterizing the second deviation, wherein x is an optimization variable and corresponds to effective initial position information.

According to an embodiment of the present disclosure, the first deviation α_i(x) The target detection frame information used for calculating the effective initial position information and the probability density function of the effective ranging result can be used for representing. According to the embodiment of the disclosure, the probability density function may be determined according to the probability distribution corresponding to the target detection frame information.

According to the embodiment of the present disclosure, a difference formula can be utilized according to 3 adjacent valid initial position information pos_m...pos_m+1、pos_m+2The position difference between the two objects is obtained to obtain the movement speed vel of the target object_mTo_m+1And vel_m+1To_m+2Then obtaining the acceleration acc of the target object_m+1The deviation of the acceleration of the target object from the prior value (acc/prior value) is used as the deviation β of the smoothing degree between the adjacent effective initial position information and the prior value_j(x_j，x_j+1，x_j+2). Wherein, the a priori value can be a preset fixed value.

According to the embodiment of the disclosure, the nonlinear optimization is performed on one or more pieces of effective initial position information to minimize the target deviation, the effective initial position information can be continuously changed, and the target deviation is iteratively solved by using a nonlinear optimization method to obtain the optimized position information of the target object.

According to the embodiment of the disclosure, the target deviation can be minimized by carrying out nonlinear optimization on the effective initial position information to obtain the corresponding optimized position information.

According to an embodiment of the present disclosure, optimizing the one or more valid initial position information may further include: determining whether the corresponding optimized position information after each effective initial position information is subjected to nonlinear optimization is abnormal; and filtering the abnormal optimized position information.

After filtering the abnormal optimized location information, performing nonlinear optimization on the remaining optimized location information again to minimize the target deviation, where each remaining optimized location information has corresponding final optimized location information after performing nonlinear optimization.

Further, the optimization speed of the target object can be calculated according to the final optimized position information corresponding to each piece of remaining optimized position information after nonlinear optimization. For example, first, several latest final optimized positions are obtained, and then the average target velocities corresponding to these latest final optimized positions are calculated using a differential algorithm (e.g., position 2 minus position 1, and then divided by the time difference). Furthermore, low-pass filtering can be performed on the average target speed to obtain a smooth and jump-free optimized speed.

According to an embodiment of the present disclosure, the first deviation may comprise a first sub-deviation and/or a second sub-deviation.

The first sub-deviation is an observed deviation between the effective initial position information and the target detection frame information used for calculating the effective initial position information. The target detection frame information used for calculating the valid initial position information may include, for example, the orientation of the imaging device, the field angle, the position, and the valid distance measurement result.

The second sub-deviation is an observed deviation between the valid initial position information and the valid ranging result used to calculate the valid initial position information.

According to the embodiment of the present disclosure, determining whether the optimized position information corresponding to each effective initial position information after the nonlinear optimization is abnormal includes: and if the detection frame information corresponding to the first sub-deviation is determined to be abnormal and/or the effective ranging result corresponding to the second sub-deviation is determined to be abnormal, determining the abnormal detection frame information and/or the optimized position corresponding to the abnormal effective ranging result as the abnormal optimized position.

According to an embodiment of the present disclosure, a manner of determining that the detection frame information corresponding to the first sub-deviation is abnormal may be as follows: calculating observation deviation between corresponding optimized position information after nonlinear optimization of the effective initial position information and target detection frame information used for calculating and obtaining the effective initial position information to obtain a first deviation value; and comparing the first deviation value with a first deviation threshold value, determining whether the first deviation value is greater than or equal to the first deviation threshold value, and if the first deviation value is greater than or equal to the first deviation threshold value, determining that the detection frame information corresponding to the first sub-deviation is abnormal.

According to an embodiment of the present disclosure, a manner of determining that the valid ranging result corresponding to the second sub-offset is abnormal may be as follows: calculating observation deviation between corresponding optimized position information after nonlinear optimization of the effective initial position information and an effective ranging result used for calculating the effective initial position information to obtain a second deviation value; and comparing the second deviation value with a second deviation threshold value, determining whether the second deviation value is greater than or equal to the second deviation threshold value, and if the second deviation value is greater than or equal to the second deviation threshold value, determining that the effective ranging result corresponding to the second sub-deviation is abnormal.

According to the embodiment of the disclosure, the optimized position information related to the abnormal detection frame information and the abnormal effective ranging result can be filtered, and then the remaining optimized position information is subjected to nonlinear optimization again to minimize the target deviation. After the target deviation is minimized, each piece of remaining optimized position information has corresponding final optimized position information after being subjected to nonlinear optimization, the final optimized position information obtained after being subjected to nonlinear optimization again is used as historical state information of the target object, and the state information of the target object, such as the position, the speed, the orientation and the like of the target object, can be predicted according to the historical state information of the target object.

Because the state information of the target object can be predicted according to the historical state information of the target object, even if the target object is not located in the center of the picture, the target distance cannot be obtained through the ranging device, and the position of the target object cannot be determined, or when an obstacle exists in the picture of the imaging device to shield the target object and the position of the target object cannot be determined after visual tracking is lost, according to the embodiment of the disclosure, the distance of the target object corresponding to the detection frame lacking an effective ranging result can be calculated according to the motion continuity of the target object and the historical state information, and the availability of the state information of the target object is improved.

According to an embodiment of the present disclosure, the first deviation threshold and the second deviation threshold may be empirically preset.

According to the embodiment of the disclosure, further, the abnormal detection frame information and/or the abnormal effective ranging result can be filtered.

Through the embodiment of the disclosure, the abnormal target detection frame can be identified and filtered, the state estimation failure of the target object caused by abnormal observation is avoided, and the reliability of the state estimation is improved. In addition, abnormal ranging results can be identified and filtered, the state estimation failure of the target object caused by abnormal observation is avoided, and the reliability of state estimation is improved.

Through the embodiment of the disclosure, the historical motion trail of the target object can be optimized, and then the motion trail of the target object is predicted based on the optimized motion trail.

According to the embodiment of the disclosure, after a new detection frame is obtained each time, if the detection frame is invalid or abnormal, the trajectory estimation of the target object can be performed on the basis of the historical motion trajectory estimation data. The invalid detection frame may include a detection frame that is obtained after the current image is identified and does not include the target object.

According to the embodiments of the present disclosure, in the case where the target object is not recognized in the obtained image, the position information when the target object is lost can be determined; and predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

According to an embodiment of the present disclosure, predicting the state information of the target object may be generating a probability distribution about a predicted position of the target object based on the position information when the target object is lost and the smoothed motion trajectory about the target object.

Target object at t₀The lost position of the moment is shown in fig. 13. According to an embodiment of the present disclosure, the spatial occupancy of the probability distribution of the predicted position of the target object increases as the loss time of the target object increases. As shown in FIG. 13, the target object is at t₁The spatial aspect of the probability distribution of the predicted position at a time is less than at t₂Spatial proportion of probability distribution of predicted position at time t of target object₂The spatial aspect of the probability distribution of the predicted position at a time is less than at t₃The spatial proportion of the probability distribution of the predicted positions of the time instants. As the loss time increases, the predicted position of the target object represented by the ellipse center continuously deviates from the lost position of the target object, and the prediction error range represented by the ellipse area continuously increases.

According to the embodiment of the disclosure, when the target object is lost, the position and the position error of the target object at any time point after the target object is lost can be predicted according to the historical track information of the target object, and the continuity of the state estimation of the target object is improved.

According to an embodiment of the present disclosure, a variation parameter of the space ratio of the probability distribution of the predicted position is related to the type of the target object, and different variation parameters may be employed for different kinds of target objects. According to an embodiment of the present disclosure, the variation parameter includes a rate of increase of a spatial proportion of the probability distribution of the predicted location.

According to the embodiment of the present disclosure, in the case where the type of the target object is a living organism, the first growth speed of the spatial proportion of the probability distribution of the predicted position is the same in different directions. For example, the organism may be a human, dog, horse, or the like.

According to an embodiment of the present disclosure, in a case where the type of the target object is a mobile device, the second growth speed of the spatial proportion of the probability distribution of the predicted position increases in the moving direction of the mobile device. For example, the mobile device may be a car, train, boat, etc.

According to an embodiment of the present disclosure, the first growth rate is less than the second growth rate. Specifically, if the target object is a human, the increase rate of the prediction error is slow, and the increase rates in the respective directions may be equal. If the target object is a vehicle or a ship, the increase speed of the prediction error is high, and the increase direction of the prediction error is mainly focused on the target motion direction.

According to an embodiment of the present disclosure, another method for determining status information of a target object is also provided, and fig. 14 schematically illustrates a flowchart of the method for determining status information of a target object according to another embodiment of the present disclosure.

In addition, the method for determining the state information of the target object provided in this embodiment may refer to some or all of the descriptions in the method for determining the state information of the target object provided in the previous embodiment. Specifically, reference may be made to the description of fig. 4 to 13 for the same or similar technical solutions, which are not described herein again.

As shown in fig. 14, the method of determining the state information of the target object includes operations S1401 to S1405.

In operation S1401, a plurality of frame images about a target object are obtained by an imaging device carried by a movable platform.

In operation S1402, each of the plurality of frame images is identified, and a plurality of detection frame information about the target object is obtained.

According to the embodiment of the present disclosure, reference may be made to the above description of fig. 3 for the description of operation S1401 and operation S1402, and details are not repeated here.

In operation S1403, one or more pieces of target detection frame information satisfying a preset condition among the plurality of pieces of detection frame information are determined.

According to the embodiment of the present disclosure, after obtaining the plurality of detection frame information about the target object, the plurality of detection frame information may be filtered, and one or more target detection frame information satisfying a preset condition may be determined from the plurality of detection frame information.

According to an embodiment of the present disclosure, the preset condition may be a condition for determining whether the detection frame information is an abnormal detection frame. For example, the preset condition may be a condition for judging whether or not the sampling time of the detection frame information is abnormal, or may be a condition for judging whether or not the physical estimated size about the target object calculated based on the detection frame information is abnormal, or the like.

In operation S1404, a plurality of ranging results between the movable platform and the target object are obtained.

According to an embodiment of the present disclosure, reference may be made to the above description of fig. 3 for the description of operation S1404, which is not repeated herein.

In operation S1405, status information of the target object is determined according to the one or more target detection frame information and the plurality of ranging results.

According to the embodiment of the disclosure, one or more pieces of target detection frame information meeting the preset condition in the plurality of pieces of detection frame information are determined, and the state information of the target object is determined according to the one or more pieces of target detection frame information and the plurality of ranging results, so that the technical problem that the state estimation of the target object is invalid due to the fact that the state information of the target object is polluted by error data is at least partially solved, and the reliability of the state information estimation is improved. By the embodiment of the disclosure, since the detection frames can be screened, the technical problem that when the picture position (for example, an object with high similarity to the target object) which does not belong to the current target object is erroneously detected as the position of the current target object, the wrong target detection frame cannot be filtered, so that the position estimation of the target object is deviated can be solved.

According to an embodiment of the present disclosure, determining state information of a target object according to one or more target detection frame information and a plurality of ranging results includes: determining a ranging result corresponding to each target detection frame information; and determining the state information of the target object according to the information of each target detection frame and the ranging result corresponding to the information of each target detection frame.

According to an embodiment of the present disclosure, the state information of the target object includes position information; the determining one or more target detection frame information satisfying a preset condition in the plurality of detection frame information includes: and screening the plurality of detection frame information according to the sampling time point corresponding to each detection frame information to obtain one or more target detection frame information.

According to the embodiment of the present disclosure, screening a plurality of detection frame information according to a sampling time point corresponding to each detection frame information includes: and screening the information of each detection frame in sequence according to the time sequence.

According to the embodiment of the disclosure, screening each detection frame information in sequence according to a time sequence includes: calculating the time difference between the current screening detection frame information and the sampling time point of the next detection frame information adjacent to the sampling time of the current screening detection frame information; comparing the time difference to a state variable threshold; if the time difference is smaller than the state variable threshold value, filtering the information of the detection frame currently being screened; and if the time difference is greater than or equal to the state variable threshold, reserving the information of the detection frame currently being screened, wherein the reserved information of the detection frame currently being screened is the information of the target detection frame.

According to an embodiment of the present disclosure, sequentially filtering each detection frame information according to a time sequence may refer to the introduction process of filtering the initial position information about the target object corresponding to each target detection frame information in fig. 11.

According to an embodiment of the present disclosure, for example, a detection frame box within a past time range is picked up_m…box_n-1、box_nThe sampling time of these detection frames is t_m…t_n-1、t_n. Will detect the frame box_m…box_n-1、box_nAs the state variable to be screened.

And calculating the time difference between each state variable to be screened and the next state variable to be screened, and removing the state variable of which the time difference with the next state variable to be screened is smaller than the threshold value of the state variable from all the state variables to be screened. The latter state variable to be screened is the latter detection frame information adjacent to the sampling time of the detection frame information currently being screened.

According to an embodiment of the present disclosure, the state variable threshold may also vary with the determined target detection box information.

According to an embodiment of the present disclosure, the state variable threshold is a time difference between the detection frame information currently being screened and a sampling time point of the target detection frame information adjacent to the sampling time of the detection frame information currently being screened.

According to the embodiment of the present disclosure, specifically, the state variable threshold changes with the determined target detection frame information, which is described with reference to fig. 12 and is not described herein again.

According to an embodiment of the present disclosure, determining a plurality of target detection frame information satisfying a preset condition among a plurality of detection frame information includes: determining whether the target object moves to meet a preset condition or not according to the information of the detection frames; and under the condition that the target object moves to meet the preset condition, determining the corresponding detection frame information when the target object moves to meet the preset condition as the target detection frame information.

According to an embodiment of the present disclosure, determining whether a target object has moved to satisfy a preset condition according to a plurality of detection frame information includes: aiming at any adjacent first detection frame information and second detection frame information in the plurality of detection frame information, obtaining first state information of the imaging device during first detection frame information acquisition and second state information of the imaging device during second detection frame information acquisition; determining a first probability distribution of initial position information about the target object corresponding to the first detection frame information according to the ranging result corresponding to the first detection frame information and the first state information; determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the ranging result and the second state information corresponding to the second detection frame information; and determining whether the target object moves according to the first probability distribution and the second probability distribution, wherein the movement meets the preset condition.

According to an embodiment of the present disclosure, determining whether a target object has moved according to a preset condition according to the first probability distribution and the second probability distribution includes: determining a spatial position with the highest probability density according to the first probability distribution and the second probability distribution; calculating a first distance from a spatial location with a highest probability density to a first probability distribution center location of the first probability distribution; calculating a second distance from the spatial position with the highest probability density to the center position of a second probability distribution of the second probability distribution; determining a probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold value, determining that the target object moves to meet a preset condition.

According to the embodiment of the present disclosure, it is determined whether the target object moves according to the information of the plurality of detection frames, which is described with reference to fig. 6 to 10, and details are not repeated herein.

According to an embodiment of the present disclosure, determining state information of a target object according to one or more target detection frame information and a plurality of ranging results includes: and optimizing effective initial position information corresponding to the one or more target detection frame information to smooth the motion trail of the target object.

According to the embodiment of the disclosure, optimizing the effective initial position information corresponding to one or more target detection frame information includes: and performing nonlinear optimization on effective initial position information corresponding to one or more pieces of target detection frame information to minimize target deviation, wherein the target deviation is related to the detection frame information and/or the ranging result, and each effective initial position information has corresponding optimized position information after being subjected to nonlinear optimization.

According to an embodiment of the present disclosure, the target deviation comprises a first deviation and/or a second deviation; the first deviation comprises an observation deviation between the effective initial position information and target detection frame information and a ranging result which are used for calculating the effective initial position information; the second deviation comprises a deviation with respect to a degree of smoothness between adjacent valid initial position information of the plurality of valid initial position information from the prior value.

According to the embodiment of the disclosure, the first deviation is characterized by a probability density function of target detection frame information and ranging results used for calculating effective initial position information.

According to the embodiment of the present disclosure, optimizing the effective initial position information corresponding to the one or more target detection frame information further includes: determining whether the corresponding optimized position information after each effective initial position information is subjected to nonlinear optimization is abnormal; and filtering the abnormal optimized position information.

According to an embodiment of the present disclosure, the target deviation comprises a first deviation comprising a first sub-deviation and/or a second sub-deviation; the first sub-deviation is an observation deviation between the effective initial position information and target detection frame information used for calculating the effective initial position information; the second sub-deviation is an observed deviation between the effective initial position information and the ranging result used for calculating the effective initial position information.

Determining whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal includes: and if the detection frame information corresponding to the first sub-deviation is determined to be abnormal and/or the distance measurement result corresponding to the second sub-deviation is determined to be abnormal, determining the abnormal detection frame information and/or the optimized position corresponding to the abnormal distance measurement result as the abnormal optimized position.

According to the embodiment of the present disclosure, optimizing the effective initial position information corresponding to the one or more target detection frame information further includes: after filtering the abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information to minimize the target deviation, wherein each remaining optimized position information has corresponding final optimized position information after being subjected to nonlinear optimization.

According to an embodiment of the present disclosure, in a case where a target object is not recognized in an obtained image, position information when the target object is lost is determined; and predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

According to the embodiment of the present disclosure, predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory of the target object includes: and generating probability distribution of the predicted position of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

According to an embodiment of the present disclosure, the spatial occupancy of the probability distribution of the predicted position increases as the loss time of the target object increases.

According to an embodiment of the present disclosure, a variation parameter of the space ratio of the probability distribution of the predicted position is related to the type of the target object.

According to an embodiment of the present disclosure, the variation parameter includes a speed of increase of a space ratio of the probability distribution of the predicted location, and the correlation of the variation parameter of the space ratio of the probability distribution of the predicted location with the type of the target object includes: in the case where the type of the target object is a living organism, first growth speeds of spatial proportions of probability distributions of the predicted positions are the same in different directions; in the case where the type of the target object is a mobile device, the second growth speed of the spatial proportion of the probability distribution of the predicted position increases in the direction of motion of the mobile device.

According to an embodiment of the present disclosure, the first growth rate is less than the second growth rate.

According to an embodiment of the present disclosure, reference may be made to the description of fig. 13 above for a process of predicting state information of a target object, and details are not repeated here.

According to an embodiment of the present disclosure, determining state information of a target object according to one or more target detection frame information and a plurality of ranging results includes: screening effective ranging results from the ranging results according to the information of the one or more target detection frames; and determining the state information of the target object according to the one or more target detection frame information and the effective ranging result.

According to an embodiment of the present disclosure, a method for screening a valid ranging result from a plurality of ranging results according to one or more target detection frame information includes: determining one or more ranging results corresponding to each piece of target detection frame information; and screening one or more ranging results corresponding to each piece of target detection frame information.

According to the embodiment of the disclosure, determining one or more ranging results corresponding to each piece of target detection frame information includes: and determining one or more ranging results corresponding to each piece of target detection frame information according to the sampling time point of each piece of target detection frame information and the sampling time point of each ranging result in the plurality of ranging results.

According to the embodiment of the present disclosure, the ranging result includes a laser ranging result, and the screening of one or more ranging results corresponding to each target detection frame information includes: determining a laser spot corresponding to each ranging result in one or more ranging results; determining the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and screening one or more ranging results corresponding to each target detection frame information according to the effectiveness of each ranging result.

According to the embodiment of the disclosure, determining the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result includes: determining the area coincidence rate of the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and determining the effectiveness of each ranging result according to the area coincidence rate.

According to the embodiment of the present disclosure, the validity of each ranging result may be determined according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, which is described with reference to fig. 5 above and is not described herein again.

According to the embodiment of the disclosure, determining the validity of each ranging result according to the area coincidence rate includes: comparing the area coincidence rate with a preset proportional threshold; determining the ranging result with the area coincidence rate larger than or equal to a preset proportion threshold value as an effective ranging result; and determining the ranging result with the area coincidence rate smaller than the preset proportion threshold value as an invalid ranging result.

According to the embodiment of the disclosure, in a case that it is determined that each piece of target detection frame information corresponds to a plurality of ranging results, determining validity of each ranging result according to the piece of target detection frame information corresponding to each ranging result and a laser spot corresponding to each ranging result includes: determining interpolation target detection frame information corresponding to each ranging result in a plurality of ranging results between sampling times of two pieces of target detection frame information adjacent to each sampling time according to the two pieces of target detection frame information adjacent to each sampling time to obtain target detection frame information corresponding to each ranging result; and determining the validity of each ranging result according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result.

According to an embodiment of the present disclosure, determining state information of a target object according to a plurality of target detection frame information and effective ranging results includes: determining an effective ranging result corresponding to each target detection frame information; and determining the state information of the target object according to the information of each target detection frame and the effective ranging result corresponding to the information of each target detection frame.

According to the embodiment of the disclosure, determining the effective ranging result corresponding to each target detection frame information includes: and associating each effective ranging result to the target detection frame information closest to the sampling time point of the effective ranging result according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result.

According to an embodiment of the present disclosure, in a case that each piece of target detection frame information corresponds to a plurality of valid ranging results, determining state information of a target object according to each piece of target detection frame information and the valid ranging result corresponding to each piece of target detection frame information includes: calculating a weighted average value of a plurality of effective ranging results corresponding to each target detection frame information to obtain a target ranging result corresponding to each target detection frame information; and determining the state information of the target object according to the information of each target detection frame and the target ranging result corresponding to the information of each target detection frame.

According to an embodiment of the present disclosure, determining state information of a target object according to one or more target detection frame information and a plurality of ranging results includes: determining a ranging result corresponding to each target detection frame information; determining a physical estimated size of the target object corresponding to each target detection frame information; screening the ranging result corresponding to each target detection frame information according to the physical estimation size of the target object corresponding to each target detection frame information; and determining the state information of the target object according to the one or more target detection frame information and the screened ranging result.

According to the embodiment of the disclosure, screening the ranging result corresponding to each target detection frame information according to the physical estimation size of the target object corresponding to each target detection frame information includes: comparing the physical estimation size of the target object corresponding to each target detection frame information with a preset reasonable range; and filtering the ranging result corresponding to the target detection frame information under the condition that the physical estimation size of the target object corresponding to the target detection frame information does not accord with the preset reasonable range.

According to the embodiment of the disclosure, determining the physical estimated size of the target object corresponding to each piece of target detection frame information includes: determining a view angle corresponding to each target detection frame information according to each target detection frame information and the view angle of the imaging device when each target detection frame information is acquired; and determining the physical estimation size corresponding to each target detection frame information according to the field angle corresponding to each target detection frame information and the ranging result corresponding to each target detection frame information.

According to an embodiment of the present disclosure, an object type of the target object may also be determined, where each object type has a corresponding preset reasonable range. Comparing the physical estimated size of the target object corresponding to each target detection frame information with a preset reasonable range comprises: determining a target preset reasonable range according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each target detection frame information with a target preset reasonable range.

According to an embodiment of the present disclosure, there is also provided a device for determining status information of a target object, including: a processor; a readable storage medium storing one or more programs, wherein the one or more programs, when executed by the processor, cause the processor to:

obtaining a plurality of frame images about a target object obtained by an imaging device carried by a movable platform; identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object; obtaining a plurality of ranging results between the movable platform and the target object; screening effective ranging results from the ranging results according to the information of the detection frames; and determining the state information of the target object according to the plurality of detection frame information and the effective ranging result.

According to the embodiment of the present disclosure, the processor screens out effective ranging results from a plurality of ranging results according to a plurality of detection frame information, including: determining one or more ranging results corresponding to each detection frame information; and screening one or more ranging results corresponding to each detection frame information.

According to an embodiment of the disclosure, the processor determines one or more ranging results corresponding to each detection frame information, including: and determining one or more ranging results corresponding to each detection frame information according to the sampling time point of each detection frame information and the sampling time point of each ranging result in the plurality of ranging results.

According to the embodiment of the present disclosure, the ranging result includes a laser ranging result, and the processor screens one or more ranging results corresponding to each detection frame information, including: determining a laser spot corresponding to each ranging result in one or more ranging results; determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and screening one or more ranging results corresponding to each detection frame information according to the effectiveness of each ranging result.

According to the embodiment of the disclosure, the processor determines the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining the coincidence rate of the area of the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and determining the effectiveness of each ranging result according to the area coincidence rate.

According to an embodiment of the disclosure, the processor determines validity of each ranging result according to the area coincidence rate, including: comparing the area coincidence rate with a preset proportional threshold; determining the ranging result with the area coincidence rate larger than or equal to a preset proportion threshold value as an effective ranging result; and determining the ranging result with the area coincidence rate smaller than the preset proportion threshold value as an invalid ranging result.

According to the embodiment of the disclosure, in a case that it is determined that each detection frame information corresponds to a plurality of ranging results, the processor determines validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining interpolation detection frame information corresponding to each ranging result in a plurality of ranging results between sampling times of two adjacent detection frame information at the sampling time according to the two adjacent detection frame information at the sampling time to obtain the detection frame information corresponding to each ranging result; and determining the validity of each ranging result according to the laser spot corresponding to each ranging result and the detection frame information corresponding to each ranging result.

According to an embodiment of the present disclosure, the processor determines the state information of the target object according to the plurality of detection frame information and the valid ranging result, including: determining an effective ranging result corresponding to each detection frame information; and determining the state information of the target object according to each detection frame information and the effective ranging result corresponding to each detection frame information.

According to the embodiment of the disclosure, the processor determines an effective ranging result corresponding to each detection frame information, including: and associating each effective ranging result to the detection frame information closest to the sampling time point of the effective ranging result according to the sampling time point of each detection frame information and the sampling time point of each effective ranging result.

According to an embodiment of the present disclosure, in a case that each detection frame information corresponds to a plurality of valid ranging results, the determining, by the processor, the state information of the target object according to each detection frame information and the valid ranging result corresponding to each detection frame information includes: calculating a weighted average value of a plurality of effective ranging results corresponding to each detection frame information to obtain a target ranging result corresponding to each detection frame information; and determining the state information of the target object according to each detection frame information and the target ranging result corresponding to each detection frame information.

According to an embodiment of the disclosure, the processor further performs the following operations: determining an effective ranging result corresponding to each detection frame information; determining a physical estimated size of the target object corresponding to each piece of detection frame information; and screening the effective ranging result corresponding to each detection frame information according to the physical estimation size of the target object corresponding to each detection frame information.

According to the embodiment of the disclosure, the processor filters the effective ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information, including: comparing the physical estimation size of the target object corresponding to each detection frame information with a preset reasonable range; and filtering the effective ranging result corresponding to the detection frame information under the condition that the physical estimation size of the target object corresponding to the detection frame information does not accord with the preset reasonable range.

According to an embodiment of the present disclosure, the processor determines a physical estimated size of the target object corresponding to each detection frame information, including: determining a field angle corresponding to each detection frame information according to each detection frame information and the field angle of the imaging device when each detection frame information is acquired; and determining the physical estimation size corresponding to each detection frame information according to the field angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.

According to an embodiment of the disclosure, the processor further performs the following operations: determining object types of the target object, wherein each object type has a corresponding preset reasonable range; comparing the physical estimation size of the target object corresponding to each detection frame information with a preset reasonable range, wherein the comparison comprises the following steps: determining a target preset reasonable range according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each detection frame information with a target preset reasonable range.

According to an embodiment of the disclosure, the processor further performs the following operations: after the effective ranging result corresponding to the detection frame information is filtered, determining the effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information with the effective ranging result filtered as the effective ranging result corresponding to the detection frame information with the effective ranging result filtered.

According to an embodiment of the present disclosure, the processor determines the state information of the target object according to the plurality of detection frame information and the valid ranging result, including: determining an effective ranging result corresponding to each piece of target detection frame information in the one or more pieces of target detection frame information, wherein the one or more pieces of target detection frame information are detection frame information meeting preset conditions in the plurality of pieces of detection frame information; and determining the state information of the target object according to the information of each target detection frame and the effective ranging result corresponding to the information of each target detection frame.

According to an embodiment of the present disclosure, the status information includes location information; the processor determines the state information of the target object according to the information of each target detection frame and the effective ranging result corresponding to the information of each target detection frame, and the method comprises the following steps: determining initial position information of the target object corresponding to each target detection frame information according to each target detection frame information and the effective ranging result corresponding to each target detection frame information to obtain a plurality of initial position information; screening the plurality of initial position information according to the sampling time point corresponding to each target detection frame information to obtain one or more effective initial position information; and determining the state information of the target object according to the one or more effective initial position information.

According to the embodiment of the disclosure, the processor screens a plurality of initial position information according to the sampling time point corresponding to each target detection frame information, including: and sequentially screening the initial position information of the target object corresponding to each piece of target detection frame information according to the time sequence.

According to the embodiment of the disclosure, the processor sequentially screens the initial position information about the target object corresponding to each piece of target detection frame information according to a time sequence, and the screening includes: calculating the time difference between the initial position information currently being screened and the sampling time point of the next initial position information adjacent to the sampling time of the initial position information currently being screened; comparing the time difference to a state variable threshold; if the time difference is smaller than the state variable threshold, filtering the initial position information currently being screened; and if the time difference is greater than or equal to the state variable threshold, reserving the initial position information currently being screened, wherein the reserved initial position information currently being screened is valid initial position information.

According to an embodiment of the present disclosure, the state variable threshold varies with the determined valid initial position information.

According to an embodiment of the disclosure, the processor further performs the following operations: determining whether the target object moves to meet a preset condition or not according to the information of the detection frames; and under the condition that the target object moves to meet the preset condition, determining the corresponding detection frame information when the target object moves to meet the preset condition as the target detection frame information.

According to an embodiment of the present disclosure, the processor determining whether the target object has moved to satisfy a preset condition according to the plurality of detection frame information includes: aiming at any adjacent first detection frame information and second detection frame information in the plurality of detection frame information, obtaining first state information of the imaging device during first detection frame information acquisition and second state information of the imaging device during second detection frame information acquisition; determining a first probability distribution of initial position information about the target object corresponding to the first detection frame information according to the effective ranging result corresponding to the first detection frame information and the first state information; determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the effective ranging result and the second state information corresponding to the second detection frame information; and determining whether the target object moves according to the first probability distribution and the second probability distribution, wherein the movement meets the preset condition.

According to an embodiment of the present disclosure, the processor determines whether the target object has moved according to a first probability distribution and a second probability distribution, including: determining a spatial position with the highest probability density according to the first probability distribution and the second probability distribution; calculating a first distance from a spatial location with a highest probability density to a first probability distribution center location of the first probability distribution; calculating a second distance from the spatial position with the highest probability density to the center position of a second probability distribution of the second probability distribution; determining a probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold value, determining that the target object moves to meet a preset condition.

According to an embodiment of the present disclosure, the processor determines the state information of the target object according to one or more valid initial position information, including: one or more valid initial position information is optimized to smooth a motion trajectory with respect to the target object.

According to an embodiment of the present disclosure, a processor optimizes one or more valid initial position information, including: and performing nonlinear optimization on one or more pieces of effective initial position information to minimize a target deviation, wherein the target deviation is related to the detection frame information and/or the effective ranging result, and each piece of effective initial position information has corresponding optimized position information after being subjected to nonlinear optimization.

According to an embodiment of the present disclosure, the target deviation comprises a first deviation and/or a second deviation; the first deviation comprises an observation deviation between the effective initial position information and target detection frame information and an effective ranging result which are used for calculating the effective initial position information; the second deviation comprises a deviation of the degree of smoothness between adjacent valid initial position information from the prior value.

According to the embodiment of the disclosure, the first deviation is characterized by a probability density function used for calculating target detection frame information and effective ranging results of obtaining effective initial position information.

According to an embodiment of the present disclosure, the processor optimizes one or more valid initial position information, further comprising: determining whether the corresponding optimized position information after each effective initial position information is subjected to nonlinear optimization is abnormal; and filtering the abnormal optimized position information.

According to an embodiment of the present disclosure, the target deviation comprises a first deviation comprising a first sub-deviation and/or a second sub-deviation; the first sub-deviation is an observation deviation between the effective initial position information and target detection frame information used for calculating the effective initial position information; the second sub-deviation is an observation deviation between the effective initial position information and an effective ranging result used for calculating the effective initial position information; wherein, the processor determines whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal, including: and if the detection frame information corresponding to the first sub-deviation is determined to be abnormal and/or the effective ranging result corresponding to the second sub-deviation is determined to be abnormal, determining the abnormal detection frame information and/or the optimized position corresponding to the abnormal effective ranging result as the abnormal optimized position.

According to an embodiment of the present disclosure, the processor optimizes one or more valid initial position information, further comprising: after filtering the abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information to minimize the target deviation, wherein each remaining optimized position information has corresponding final optimized position information after being subjected to nonlinear optimization.

According to an embodiment of the disclosure, the processor further performs the following operations: determining position information when the target object is lost in the case where the target object is not recognized in the obtained image; and predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

According to an embodiment of the present disclosure, the predicting, by the processor, the state information of the target object according to the position information when the target object is lost and the smoothed motion trajectory about the target object includes: and generating probability distribution of the predicted position of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

According to an embodiment of the present disclosure, the variation parameter includes a speed of increase of a space ratio of the probability distribution of the predicted location; in the case where the type of the target object is a living organism, first growth speeds of spatial proportions of probability distributions of the predicted positions are the same in different directions; in the case where the type of the target object is a mobile device, the second growth speed of the spatial proportion of the probability distribution of the predicted position increases in the direction of motion of the mobile device.

According to an embodiment of the present disclosure, there is also provided another apparatus for determining status information of a target object, including: a processor; a readable storage medium storing one or more programs, wherein the one or more programs, when executed by the processor, cause the processor to:

obtaining a plurality of frame images about a target object obtained by an imaging device carried by a movable platform; identifying each frame of image in the multi-frame images to obtain a plurality of pieces of detection frame information about the target object; determining one or more target detection frame information satisfying a preset condition in the plurality of detection frame information; obtaining a plurality of ranging results between the movable platform and the target object; and determining the state information of the target object according to the one or more target detection frame information and the plurality of ranging results.

According to an embodiment of the present disclosure, the processor determines state information of a target object according to one or more target detection frame information and a plurality of ranging results, including: determining a ranging result corresponding to each target detection frame information; and determining the state information of the target object according to the information of each target detection frame and the ranging result corresponding to the information of each target detection frame.

According to an embodiment of the present disclosure, the status information includes location information; the method for determining one or more target detection frame information meeting a preset condition in the plurality of detection frame information by the processor comprises the following steps: and screening the plurality of detection frame information according to the sampling time point corresponding to each detection frame information to obtain one or more target detection frame information.

According to the embodiment of the disclosure, the processor screens a plurality of detection frame information according to the sampling time point corresponding to each detection frame information, including: and screening the information of each detection frame in sequence according to the time sequence.

According to the embodiment of the disclosure, the processor sequentially filters each detection frame information according to a time sequence, including: calculating the time difference between the current screening detection frame information and the sampling time point of the next detection frame information adjacent to the sampling time of the current screening detection frame information; comparing the time difference to a state variable threshold; if the time difference is smaller than the state variable threshold value, filtering the information of the detection frame currently being screened; and if the time difference is greater than or equal to the state variable threshold, reserving the information of the detection frame currently being screened, wherein the reserved information of the detection frame currently being screened is the information of the target detection frame.

According to the embodiment of the disclosure, the state variable threshold value is changed along with the determined target detection frame information.

According to an embodiment of the present disclosure, the processor determining a plurality of target detection frame information satisfying a preset condition among the plurality of detection frame information includes: determining whether the target object moves to meet a preset condition or not according to the information of the detection frames; and under the condition that the target object moves to meet the preset condition, determining the corresponding detection frame information when the target object moves to meet the preset condition as the target detection frame information.

According to an embodiment of the present disclosure, the processor determining whether the target object has moved to satisfy a preset condition according to the plurality of detection frame information includes: aiming at any adjacent first detection frame information and second detection frame information in the plurality of detection frame information, obtaining first state information of the imaging device during first detection frame information acquisition and second state information of the imaging device during second detection frame information acquisition; determining a first probability distribution of initial position information about the target object corresponding to the first detection frame information according to the ranging result corresponding to the first detection frame information and the first state information; determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the ranging result and the second state information corresponding to the second detection frame information; and determining whether the target object moves according to the first probability distribution and the second probability distribution, wherein the movement meets the preset condition.

According to an embodiment of the present disclosure, the processor determining whether the target object has moved according to the first probability distribution and the second probability distribution includes: determining a spatial position with the highest probability density according to the first probability distribution and the second probability distribution; calculating a first distance from a spatial location with a highest probability density to a first probability distribution center location of the first probability distribution; calculating a second distance from the spatial position with the highest probability density to the center position of a second probability distribution of the second probability distribution; determining a probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and if the probability distance is greater than or equal to a preset threshold value, determining that the target object moves to meet a preset condition.

According to an embodiment of the present disclosure, the processor determines state information of a target object according to one or more target detection frame information and a plurality of ranging results, including: and optimizing effective initial position information corresponding to the one or more target detection frame information to smooth the motion trail of the target object.

According to the embodiment of the disclosure, the processor optimizes effective initial position information corresponding to one or more target detection frame information, including: and performing nonlinear optimization on effective initial position information corresponding to one or more pieces of target detection frame information to minimize target deviation, wherein the target deviation is related to the detection frame information and/or the ranging result, and each effective initial position information has corresponding optimized position information after being subjected to nonlinear optimization.

According to the embodiment of the disclosure, the processor optimizes the effective initial position information corresponding to the one or more target detection frame information, and further includes: determining whether the corresponding optimized position information after each effective initial position information is subjected to nonlinear optimization is abnormal; and filtering the abnormal optimized position information.

According to an embodiment of the present disclosure, the target deviation comprises a first deviation comprising a first sub-deviation and/or a second sub-deviation; the first sub-deviation is an observation deviation between the effective initial position information and target detection frame information used for calculating the effective initial position information; the second sub-deviation is an observation deviation between the effective initial position information and a ranging result used for calculating the effective initial position information; wherein, the processor determines whether the optimized position information corresponding to each effective initial position information after nonlinear optimization is abnormal, including: and if the detection frame information corresponding to the first sub-deviation is determined to be abnormal and/or the distance measurement result corresponding to the second sub-deviation is determined to be abnormal, determining the abnormal detection frame information and/or the optimized position corresponding to the abnormal distance measurement result as the abnormal optimized position.

According to the embodiment of the disclosure, the processor optimizes the effective initial position information corresponding to the one or more target detection frame information, and further includes: after filtering the abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information to minimize the target deviation, wherein each remaining optimized position information has corresponding final optimized position information after being subjected to nonlinear optimization.

According to the embodiment of the disclosure, the predicting the state information of the target object by the processor according to the position information when the target object is lost and the smoothed motion trail of the target object comprises: and generating probability distribution of the predicted position of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

According to an embodiment of the present disclosure, the processor determines state information of a target object according to one or more target detection frame information and a plurality of ranging results, including: screening effective ranging results from the ranging results according to the information of the one or more target detection frames; and determining the state information of the target object according to the one or more target detection frame information and the effective ranging result.

According to an embodiment of the present disclosure, the processor selects a valid ranging result from the plurality of ranging results according to the one or more target detection frame information, including: determining one or more ranging results corresponding to each piece of target detection frame information; and screening one or more ranging results corresponding to each piece of target detection frame information.

According to the embodiment of the disclosure, the processor determines one or more ranging results corresponding to each piece of target detection frame information, including: and determining one or more ranging results corresponding to each piece of target detection frame information according to the sampling time point of each piece of target detection frame information and the sampling time point of each ranging result in the plurality of ranging results.

According to the embodiment of the present disclosure, the ranging result includes a laser ranging result, and the processor screens one or more ranging results corresponding to each target detection frame information, including: determining a laser spot corresponding to each ranging result in one or more ranging results; determining the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and screening one or more ranging results corresponding to each target detection frame information according to the effectiveness of each ranging result.

According to the embodiment of the disclosure, the processor determines validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, including: determining the area coincidence rate of the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and determining the effectiveness of each ranging result according to the area coincidence rate.

According to an embodiment of the present disclosure, the processor determining the validity of each ranging result according to the area coincidence rate includes: comparing the area coincidence rate with a preset proportional threshold; determining the ranging result with the area coincidence rate larger than or equal to a preset proportion threshold value as an effective ranging result; and determining the ranging result with the area coincidence rate smaller than the preset proportion threshold value as an invalid ranging result.

According to an embodiment of the disclosure, in a case that it is determined that each piece of target detection frame information corresponds to a plurality of ranging results, the determining, by the processor, validity of each ranging result according to the piece of target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result includes: determining interpolation target detection frame information corresponding to each ranging result in a plurality of ranging results between sampling times of two pieces of target detection frame information adjacent to each sampling time according to the two pieces of target detection frame information adjacent to each sampling time to obtain target detection frame information corresponding to each ranging result; and determining the validity of each ranging result according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result.

According to an embodiment of the present disclosure, the determining, by the processor, the state information of the target object according to the plurality of target detection frame information and the valid ranging result includes: determining an effective ranging result corresponding to each target detection frame information; and determining the state information of the target object according to the information of each target detection frame and the effective ranging result corresponding to the information of each target detection frame.

According to the embodiment of the disclosure, the determining, by the processor, the effective ranging result corresponding to each piece of target detection frame information includes: and associating each effective ranging result to the target detection frame information closest to the sampling time point of the effective ranging result according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result.

According to an embodiment of the present disclosure, in a case that each piece of target detection frame information corresponds to a plurality of valid ranging results, the determining, by the processor, the state information of the target object according to each piece of target detection frame information and the valid ranging result corresponding to each piece of target detection frame information includes: calculating a weighted average value of a plurality of effective ranging results corresponding to each target detection frame information to obtain a target ranging result corresponding to each target detection frame information; and determining the state information of the target object according to the information of each target detection frame and the target ranging result corresponding to the information of each target detection frame.

According to an embodiment of the present disclosure, the processor determines state information of a target object according to one or more target detection frame information and a plurality of ranging results, including: determining a ranging result corresponding to each target detection frame information; determining a physical estimated size of the target object corresponding to each target detection frame information; screening the ranging result corresponding to each target detection frame information according to the physical estimation size of the target object corresponding to each target detection frame information; and determining the state information of the target object according to the one or more target detection frame information and the screened ranging result.

According to the embodiment of the disclosure, the screening, by the processor, of the ranging result corresponding to each piece of target detection frame information according to the physical estimated size of the target object corresponding to each piece of target detection frame information includes: comparing the physical estimation size of the target object corresponding to each target detection frame information with a preset reasonable range; and filtering the ranging result corresponding to the target detection frame information under the condition that the physical estimation size of the target object corresponding to the target detection frame information does not accord with the preset reasonable range.

According to an embodiment of the present disclosure, the processor determines a physical estimated size of the target object corresponding to each target detection frame information, including: determining a view angle corresponding to each target detection frame information according to each target detection frame information and the view angle of the imaging device when each target detection frame information is acquired; and determining the physical estimation size corresponding to each target detection frame information according to the field angle corresponding to each target detection frame information and the ranging result corresponding to each target detection frame information.

According to an embodiment of the disclosure, the processor further performs the following operations: determining object types of the target object, wherein each object type has a corresponding preset reasonable range; wherein, comparing the physical estimation size of the target object corresponding to each target detection frame information with a preset reasonable range comprises: determining a target preset reasonable range according to the object type of the target object; and comparing the physical estimated size of the target object corresponding to each target detection frame information with a target preset reasonable range.

In particular, processors may comprise, for example, a general purpose microprocessor, an instruction set processor and/or related chip set and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like, in accordance with embodiments of the present disclosure. The processor may also include on-board memory for caching purposes. The processor may be a single processing unit or a plurality of processing units for performing the different actions of the method flows according to embodiments of the present disclosure.

According to an embodiment of the present disclosure, there is also provided a system for determining state information of a target object, including: an imaging apparatus for obtaining a plurality of frame images about a target object and a state information determination apparatus mentioned in the above-described embodiments.

There is also provided, in accordance with an embodiment of the present disclosure, a movable platform, including: a movable body and a target object.

It should be noted that the present disclosure omits some descriptions of well-known components of the movable platform, as would be apparent to those skilled in the art. The movable platform may have different components, as this is for different equipment situations. For example, when the movable platform is an unmanned aerial vehicle, a rotor, a rotating mechanism and the like can be further included, and the description of the rotor, the rotating mechanism and the like is omitted in the present disclosure. When the movable platform is an unmanned vehicle, the movable platform can also comprise an engine, wheels and the like, and the description of the engine, the wheels and the like is omitted in the disclosure.

According to an embodiment of the present disclosure, there is also provided a readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the state information determination method mentioned in the above-mentioned embodiment.

The readable storage medium may be included in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The readable storage medium carries one or more programs which, when executed, implement a method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the readable storage medium may be a non-volatile readable storage medium, which may include, for example but not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

It should be noted that the state information determination system 1500 of the target object may also have some or all of the hardware modules shown in fig. 15.

As shown in fig. 15, a state information determination system 1500 of a target object according to an embodiment of the present disclosure includes a processor 1501 which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1502 or a program loaded from a storage section 1508 into a Random Access Memory (RAM) 1503. Processor 1501 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset(s) and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), and so forth. The processor 1501 may also include on-board memory for caching purposes. Processor 1501 may include a single processing unit or multiple processing units for performing different acts of a method flow in accordance with embodiments of the present disclosure.

In the RAM 1503, various programs and data necessary for the operation of the state information determination system 1500 of the target object are stored. The processor 1501, the ROM 1502, and the RAM 1503 are connected to each other by a bus 1504. The processor 1501 executes various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1502 and/or RAM 1503. Note that the programs may also be stored in one or more memories other than the ROM 1502 and RAM 1503. The processor 1501 may also execute various operations of the method flows according to the embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the target object's state information determination system 1500 may also include an input/output (I/O) interface 1505, the input/output (I/O) interface 1505 also being connected to the bus 1504. The target object's state information determination system 1500 may also include one or more of the following components connected to the I/O interface 1505: an input portion 1506 including a keyboard, a mouse, and the like; an output portion 1507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1508 including a hard disk and the like; and a communication section 1509 including a network interface card such as a LAN card, a modem, or the like. The communication section 1509 performs communication processing via a network such as the internet. A drive 1510 is also connected to the I/O interface 1505 as needed. A removable medium 1511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1510 as necessary, so that a computer program read out therefrom is mounted into the storage section 1508 as necessary.

According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 1509, and/or installed from the removable medium 1511. The computer program, when executed by the processor 1501, performs the above-described functions defined in the system of the embodiments of the present disclosure.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not described herein again.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present disclosure, and not for limiting the same; while the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; features in embodiments of the disclosure may be combined arbitrarily, without conflict; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present disclosure.

Claims

1. A method for determining state information of a target object, comprising:

obtaining a plurality of frames of images about the target object by an imaging device carried by a movable platform;

identifying each frame of image in a plurality of frames of images to obtain a plurality of pieces of detection frame information about the target object;

obtaining a plurality of ranging results between the movable platform and the target object;

screening out effective ranging results from the ranging results according to the information of the detection frames; and

and determining the state information of the target object according to the plurality of detection frame information and the effective ranging result.

2. The method of claim 1, wherein the selecting a valid ranging result from the plurality of ranging results according to the plurality of detection frame information comprises:

determining one or more ranging results corresponding to each piece of detection frame information; and

and screening one or more ranging results corresponding to each piece of detection frame information.

3. The method of claim 2, wherein the determining one or more ranging results corresponding to each detection frame information comprises:

and determining one or more ranging results corresponding to each piece of detection frame information according to the sampling time point of each piece of detection frame information and the sampling time point of each ranging result in the plurality of ranging results.

4. The method of claim 2, wherein the ranging results comprise laser ranging results, and the screening one or more ranging results corresponding to each detection frame information comprises:

determining a laser spot corresponding to each ranging result in one or more ranging results;

determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and

and screening one or more ranging results corresponding to each detection frame information according to the effectiveness of each ranging result.

5. The method according to claim 4, wherein the determining the validity of each ranging result according to the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result comprises:

determining the area coincidence rate of the detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and

and determining the effectiveness of each ranging result according to the area coincidence rate.

6. The method of claim 5, wherein determining the validity of each ranging result according to the area coincidence ratio comprises:

comparing the area coincidence rate with a preset proportion threshold;

determining the ranging result with the area coincidence rate larger than or equal to the preset proportion threshold value as an effective ranging result; and

and determining the ranging result with the area coincidence rate smaller than the preset proportion threshold value as an invalid ranging result.

7. The method according to claim 4, wherein in a case that it is determined that each of the detection frame information corresponds to a plurality of ranging results, the determining validity of each of the ranging results according to the detection frame information corresponding to each of the ranging results and the laser spot corresponding to each of the ranging results comprises:

according to the two pieces of detection frame information with adjacent sampling time, determining interpolation detection frame information corresponding to each ranging result in the plurality of ranging results between the sampling time of the two pieces of detection frame information with adjacent sampling time, and obtaining the detection frame information corresponding to each ranging result; and

and determining the validity of each ranging result according to the laser spot corresponding to each ranging result and the detection frame information corresponding to each ranging result.

8. The method of claim 1, wherein determining the status information of the target object according to the plurality of detection frame information and the valid ranging result comprises:

determining an effective ranging result corresponding to each detection frame information; and

and determining the state information of the target object according to each detection frame information and the effective ranging result corresponding to each detection frame information.

9. The method of claim 8, wherein the determining the valid ranging result corresponding to each detection frame information comprises:

and associating each effective ranging result to the detection frame information closest to the sampling time point of the effective ranging result according to the sampling time point of each detection frame information and the sampling time point of each effective ranging result.

10. The method according to claim 8, wherein in a case that each of the detection frame information corresponds to a plurality of valid ranging results, the determining the state information of the target object according to each of the detection frame information and the valid ranging result corresponding to each of the detection frame information comprises:

calculating a weighted average value of a plurality of effective ranging results corresponding to each detection frame information to obtain a target ranging result corresponding to each detection frame information; and

and determining the state information of the target object according to each piece of detection frame information and the target ranging result corresponding to each piece of detection frame information.

11. The method of claim 1, further comprising:

determining an effective ranging result corresponding to each detection frame information;

determining a physical estimated size of the target object corresponding to each piece of detection frame information; and

and screening the effective ranging result corresponding to each detection frame information according to the physical estimation size of the target object corresponding to each detection frame information.

12. The method according to claim 11, wherein the screening the valid ranging result corresponding to each detection frame information according to the physical estimated size of the target object corresponding to each detection frame information comprises:

comparing the physical estimation size of the target object corresponding to each detection frame information with a preset reasonable range; and

and filtering the effective ranging result corresponding to the detection frame information under the condition that the physical estimation size of the target object corresponding to the detection frame information does not accord with the preset reasonable range.

13. The method of claim 12, wherein the determining the physical estimated size of the target object corresponding to each of the detection frame information comprises:

determining a field angle corresponding to each piece of detection frame information according to each piece of detection frame information and the field angle of the imaging device when each piece of detection frame information is acquired; and

and determining the physical estimation size corresponding to each detection frame information according to the field angle corresponding to each detection frame information and the effective ranging result corresponding to each detection frame information.

14. The method of claim 12, further comprising:

determining object types of the target object, wherein each object type has a corresponding preset reasonable range;

the comparing the physical estimated size of the target object corresponding to each piece of the detection frame information with a preset reasonable range includes:

determining a target preset reasonable range according to the object type of the target object; and

and comparing the physical estimation size of the target object corresponding to each piece of detection frame information with the target preset reasonable range.

15. The method according to claim 12, further comprising, after filtering out valid ranging results corresponding to the detection frame information:

and determining the effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information with the effective ranging result filtered out as the effective ranging result corresponding to the detection frame information with the effective ranging result filtered out.

16. The method of claim 1, wherein determining the status information of the target object according to the plurality of detection frame information and the valid ranging result comprises:

determining an effective ranging result corresponding to each piece of target detection frame information in one or more pieces of target detection frame information, wherein the one or more pieces of target detection frame information are detection frame information meeting preset conditions in the plurality of pieces of detection frame information; and

and determining the state information of the target object according to the information of each target detection frame and the effective ranging result corresponding to the information of each target detection frame.

17. The method of claim 16, wherein the status information comprises location information; wherein the determining the state information of the target object according to each piece of target detection frame information and the effective ranging result corresponding to each piece of target detection frame information includes:

determining initial position information related to the target object corresponding to each target detection frame information according to each target detection frame information and an effective ranging result corresponding to each target detection frame information to obtain a plurality of initial position information;

screening the initial position information according to the sampling time point corresponding to each target detection frame information to obtain one or more effective initial position information; and

and determining the state information of the target object according to one or more effective initial position information.

18. The method according to claim 17, wherein the screening the plurality of initial position information according to the sampling time point corresponding to each target detection frame information includes:

and sequentially screening the initial position information of the target object corresponding to each piece of target detection frame information according to a time sequence.

19. The method according to claim 18, wherein the sequentially screening the initial position information about the target object corresponding to each piece of the target detection frame information in time order comprises:

calculating the time difference between the initial position information currently being screened and the sampling time point of the next initial position information adjacent to the sampling time of the initial position information currently being screened;

comparing the time difference to a state variable threshold;

if the time difference is smaller than the state variable threshold value, filtering the initial position information currently being screened; and

and if the time difference is greater than or equal to the state variable threshold, reserving the initial position information currently being screened, wherein the reserved initial position information currently being screened is the effective initial position information.

20. The method of claim 19, wherein the state variable threshold is a function of the determined valid initial position information.

21. The method of claim 20, wherein the state variable threshold is a time difference between the current initial position information being filtered and a sampling time point of valid initial position information adjacent to the sampling time of the current initial position information being filtered.

22. The method of claim 16, further comprising:

determining whether the target object moves to meet a preset condition or not according to the detection frame information; and

and under the condition that the target object moves to meet a preset condition, determining the corresponding detection frame information when the target object moves to meet the preset condition as the target detection frame information.

23. The method according to claim 22, wherein the determining whether the target object moves according to a plurality of the detection frame information, the movement satisfying a preset condition, comprises:

aiming at any adjacent first detection frame information and second detection frame information in a plurality of detection frame information, obtaining first state information of the imaging device during the acquisition of the first detection frame information and second state information of the imaging device during the acquisition of the second detection frame information;

determining a first probability distribution of initial position information about the target object corresponding to the first detection frame information according to the effective ranging result corresponding to the first detection frame information and the first state information;

determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the effective ranging result corresponding to the second detection frame information and the second state information; and

and determining whether the target object moves according to the first probability distribution and the second probability distribution, wherein the movement meets preset conditions.

24. The method of claim 23, wherein said determining whether the target object has moved according to the first probability distribution and the second probability distribution comprises:

determining a spatial position with the highest probability density according to the first probability distribution and the second probability distribution;

calculating a first distance of the spatial location with the highest probability density from a first probability distribution center location of the first probability distribution;

calculating a second distance from the spatial position with the highest probability density to the center position of a second probability distribution of the second probability distribution;

determining a probability distance between the first detection frame information and the second detection frame information according to the first distance and the second distance; and

and if the probability distance is greater than or equal to a preset threshold value, determining that the target object moves to meet a preset condition.

25. The method of claim 17, wherein determining the state information of the target object based on the one or more valid initial position information comprises:

optimizing one or more of the valid initial position information to smooth a motion trajectory with respect to the target object.

26. The method of claim 25, wherein optimizing one or more of the valid initial position information comprises:

and performing nonlinear optimization on one or more pieces of effective initial position information to minimize a target deviation, wherein the target deviation is related to the detection frame information and/or the effective ranging result, and each piece of effective initial position information has corresponding optimized position information after being subjected to nonlinear optimization.

27. The method of claim 26, wherein the target deviation comprises a first deviation and/or a second deviation;

the first deviation comprises an observed deviation between the effective initial position information and target detection frame information and effective ranging results used for calculating the effective initial position information;

the second deviation includes a deviation about a degree of smoothness between adjacent effective initial position information and a prior value.

28. The method of claim 27, wherein the first deviation is characterized by a probability density function of target detection frame information and valid ranging results used to calculate the valid initial position information.

29. The method of claim 27, wherein optimizing one or more of the valid initial position information further comprises:

determining whether the corresponding optimized position information after each effective initial position information is subjected to nonlinear optimization is abnormal; and

and filtering the abnormal optimized position information.

30. The method according to claim 29, wherein the target deviation comprises the first deviation, the first deviation comprising a first sub-deviation and/or a second sub-deviation;

the first sub-deviation is an observation deviation between the effective initial position information and target detection frame information used for calculating the effective initial position information;

the second sub-deviation is an observed deviation between the effective initial position information and an effective ranging result used for calculating the effective initial position information;

wherein, the determining whether the optimized position information corresponding to each effective initial position information after the nonlinear optimization is abnormal includes:

and if the detection frame information corresponding to the first sub-deviation is determined to be abnormal and/or the effective ranging result corresponding to the second sub-deviation is determined to be abnormal, determining the abnormal detection frame information and/or the optimized position corresponding to the abnormal effective ranging result as the abnormal optimized position.

31. The method of claim 29, wherein optimizing one or more of the valid initial position information further comprises:

after filtering the abnormal optimized position information, performing nonlinear optimization on the remaining optimized position information to minimize the target deviation, wherein each remaining optimized position information has corresponding final optimized position information after being subjected to nonlinear optimization.

32. The method of claim 25, further comprising:

determining position information when the target object is lost in the case where the target object is not recognized in the obtained image; and

and predicting the state information of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

33. The method of claim 32, wherein the predicting the state information of the target object according to the position information of the target object when the target object is lost and the smoothed motion trail of the target object comprises:

and generating probability distribution of the predicted position of the target object according to the position information when the target object is lost and the smoothed motion trail of the target object.

34. The method of claim 33, wherein the spatial fraction of the probability distribution of the predicted location increases as the time to loss of the target object increases.

35. The method of claim 34, wherein the parameter of the change in the spatial aspect of the probability distribution of predicted positions is related to the type of the target object.

36. The method of claim 35, wherein the variation parameter comprises a rate of increase of a spatial proportion of the probability distribution of the predicted location;

in a case where the type of the target object is a living organism, first growth speeds of spatial proportions of the probability distribution of the predicted position are the same in different directions;

in a case where the type of the target object is a mobile device, a second growth speed of a spatial proportion of the probability distribution of the predicted position increases in a direction of motion of the mobile device.

37. The method of claim 36, wherein the first rate of increase is less than the second rate of increase.

38. A method for determining state information of a target object, comprising:

determining one or more target detection frame information satisfying a preset condition in the plurality of detection frame information;

obtaining a plurality of ranging results between the movable platform and the target object; and

and determining the state information of the target object according to one or more pieces of target detection frame information and a plurality of ranging results.

39. The method of claim 38, wherein determining the status information of the target object according to the one or more target detection frame information and the plurality of ranging results comprises:

determining a ranging result corresponding to each piece of target detection frame information; and

and determining the state information of the target object according to each piece of target detection frame information and the ranging result corresponding to each piece of target detection frame information.

40. The method of claim 39, wherein the status information comprises location information; wherein the determining one or more target detection frame information satisfying a preset condition from among the plurality of detection frame information includes:

and screening the plurality of detection frame information according to the sampling time point corresponding to each detection frame information to obtain one or more target detection frame information.

41. The method according to claim 40, wherein the screening the plurality of detection frame information according to the sampling time point corresponding to each detection frame information comprises:

and screening the information of each detection frame in sequence according to the time sequence.

42. The method of claim 41, wherein the screening each of the detection frame information in sequence according to the time sequence comprises:

calculating the time difference between the current screening detection frame information and the sampling time point of the next detection frame information adjacent to the sampling time of the current screening detection frame information;

comparing the time difference to a state variable threshold;

if the time difference is smaller than the state variable threshold value, filtering the information of the detection frame currently being screened; and

and if the time difference is greater than or equal to the state variable threshold, reserving the currently-screened detection frame information, wherein the reserved currently-screened detection frame information is target detection frame information.

43. The method of claim 42, wherein the state variable threshold is varied with the determined target detection box information.

44. The method of claim 43, wherein the state variable threshold is a time difference between the current screening test frame information and a sampling time point of the target test frame information adjacent to the sampling time of the current screening test frame information.

45. The method according to claim 38, wherein the determining a plurality of target detection frame information satisfying a preset condition from among the plurality of detection frame information comprises:

46. The method according to claim 35, wherein the determining whether the target object moves according to a plurality of the detection frame information, the movement satisfying a preset condition, comprises:

determining a first probability distribution of initial position information about the target object corresponding to the first detection frame information according to the ranging result corresponding to the first detection frame information and the first state information;

determining a second probability distribution of the initial position information of the target object corresponding to the second detection frame information according to the ranging result corresponding to the second detection frame information and the second state information; and

47. The method according to claim 46, wherein the determining whether the target object has moved according to the first probability distribution and the second probability distribution comprises:

48. The method of claim 40, wherein the determining the state information of the target object according to the one or more target detection frame information and the plurality of ranging results comprises:

and optimizing effective initial position information corresponding to one or more target detection frame information to smooth the motion trail of the target object.

49. The method of claim 48, wherein optimizing valid initial position information corresponding to one or more of the target detection frame information comprises:

and performing nonlinear optimization on effective initial position information corresponding to one or more target detection frame information to minimize target deviation, wherein the target deviation is related to the detection frame information and/or a ranging result, and each effective initial position information has corresponding optimized position information after being subjected to nonlinear optimization.

50. The method of claim 49, wherein the target deviation comprises a first deviation and/or a second deviation;

the first deviation comprises an observed deviation between the effective initial position information and target detection frame information and a ranging result which are used for calculating the effective initial position information;

the second deviation comprises a deviation with respect to a degree of smoothness between adjacent effective initial position information of the plurality of effective initial position information from a prior value.

51. The method of claim 50, wherein the first deviation is characterized by a probability density function of target detection frame information and ranging results used for calculating the valid initial position information.

52. The method of claim 50, wherein the optimizing the valid initial position information corresponding to the one or more target detection frame information further comprises:

and filtering the abnormal optimized position information.

53. The method according to claim 52, wherein the target deviation comprises the first deviation, the first deviation comprising a first sub-deviation and/or a second sub-deviation;

the second sub-deviation is an observed deviation between the effective initial position information and a ranging result used for calculating the effective initial position information;

and if the detection frame information corresponding to the first sub-deviation is determined to be abnormal and/or the distance measurement result corresponding to the second sub-deviation is determined to be abnormal, determining the abnormal detection frame information and/or the optimized position corresponding to the abnormal distance measurement result as the abnormal optimized position.

54. The method of claim 52, wherein the optimizing the valid initial position information corresponding to the one or more target detection frame information further comprises:

55. The method of claim 9, further comprising:

56. The method of claim 55, wherein predicting the state information of the target object according to the position information of the target object when the target object is lost and the smoothed motion trail of the target object comprises:

57. The method of claim 56, wherein a spatial occupancy ratio of the probability distribution of the predicted location increases as a loss time of the target object increases.

58. The method of claim 57, wherein a parameter of a variation of a spatial proportion of the probability distribution of predicted positions is related to a type of the target object.

59. The method of claim 58, wherein the variation parameter comprises a speed of increase of a spatial proportion of the probability distribution of the predicted location, and wherein the correlation of the variation parameter of the spatial proportion of the probability distribution of the predicted location with the type of the target object comprises:

60. The method of claim 59, wherein said first rate of increase is less than said second rate of increase.

61. The method of claim 1, wherein determining the status information of the target object according to one or more target detection frame information and a plurality of ranging results comprises:

screening effective ranging results from the ranging results according to the one or more pieces of target detection frame information; and

and determining the state information of the target object according to one or more target detection frame information and the effective ranging result.

62. The method of claim 61, wherein the screening the plurality of ranging results for valid ranging results according to one or more target detection block information comprises:

determining one or more ranging results corresponding to each piece of target detection frame information; and

and screening one or more ranging results corresponding to each piece of target detection frame information.

63. The method of claim 62, wherein the determining one or more ranging results corresponding to each piece of target detection frame information comprises:

and determining one or more ranging results corresponding to each piece of target detection frame information according to the sampling time point of each piece of target detection frame information and the sampling time point of each ranging result in the plurality of ranging results.

64. The method of claim 62, wherein the ranging results comprise laser ranging results, and the screening one or more ranging results corresponding to each target detection frame information comprises:

determining the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and

and screening one or more ranging results corresponding to the target detection frame information according to the effectiveness of each ranging result.

65. The method as claimed in claim 64, wherein the determining the validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result comprises:

determining the area coincidence rate of the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result; and

66. The method of claim 65, wherein said determining the validity of each of said ranging results according to said area coincidence ratio comprises:

comparing the area coincidence rate with a preset proportion threshold;

67. The method as claimed in claim 64, wherein in the case that it is determined that each target detection frame information corresponds to a plurality of ranging results, said determining validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result comprises:

determining interpolation target detection frame information corresponding to each ranging result in a plurality of ranging results between sampling times of two pieces of target detection frame information adjacent to each other at the sampling time according to the two pieces of target detection frame information adjacent to each other at the sampling time, and obtaining target detection frame information corresponding to each ranging result; and

and determining the validity of each ranging result according to the laser spot corresponding to each ranging result and the target detection frame information corresponding to each ranging result.

68. The method of claim 61, wherein the determining the state information of the target object according to the plurality of target detection frame information and the valid ranging result comprises:

determining an effective ranging result corresponding to each piece of target detection frame information; and

69. The method of claim 68, wherein the determining the valid ranging result corresponding to each target detection frame information comprises:

and associating each effective ranging result to the target detection frame information closest to the sampling time point of the effective ranging result according to the sampling time point of each target detection frame information and the sampling time point of each effective ranging result.

70. The method of claim 68, wherein in a case that each target detection frame information corresponds to a plurality of valid ranging results, the determining the state information of the target object according to each target detection frame information and the valid ranging result corresponding to each target detection frame information comprises:

calculating a weighted average value of a plurality of effective ranging results corresponding to each piece of target detection frame information to obtain a target ranging result corresponding to each piece of target detection frame information; and

and determining the state information of the target object according to the target detection frame information and the target ranging result corresponding to the target detection frame information.

71. The method of claim 1, wherein determining the status information of the target object according to one or more target detection frame information and a plurality of ranging results comprises:

determining a ranging result corresponding to each piece of target detection frame information;

determining a physical estimated size of the target object corresponding to each piece of target detection frame information;

screening a ranging result corresponding to each target detection frame information according to the physical estimation size of the target object corresponding to each target detection frame information; and

and determining the state information of the target object according to the one or more target detection frame information and the screened ranging result.

72. The method as claimed in claim 71, wherein the screening the ranging result corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information comprises:

comparing the physical estimation size of the target object corresponding to each piece of target detection frame information with a preset reasonable range; and

and under the condition that the physical estimation size of the target object corresponding to the target detection frame information does not accord with the preset reasonable range, filtering the ranging result corresponding to the target detection frame information.

73. The method of claim 72, wherein the determining the physical estimated size of the target object corresponding to each of the target detection box information comprises:

determining a field angle corresponding to each piece of target detection frame information according to each piece of target detection frame information and the field angle of the imaging device when each piece of target detection frame information is acquired; and

and determining the physical estimation size corresponding to each piece of target detection frame information according to the field angle corresponding to each piece of target detection frame information and the ranging result corresponding to each piece of target detection frame information.

74. The method of claim 72, further comprising:

wherein comparing the physical estimated size of the target object corresponding to each piece of target detection frame information with a preset reasonable range comprises:

and comparing the physical estimation size of the target object corresponding to each piece of target detection frame information with the target preset reasonable range.

75. An apparatus for determining status information of a target object, comprising:

a processor;

a readable storage medium for storing one or more programs,

wherein the one or more programs, when executed by the processor, cause the processor to:

obtaining a plurality of frames of images about the target object obtained by an imaging device carried by a movable platform;

76. The apparatus of claim 75, wherein the processor selects a valid ranging result from the plurality of ranging results according to the plurality of detection frame information, comprising:

77. The apparatus as claimed in claim 76, wherein the processor determines one or more ranging results corresponding to each of the detection block information, comprising:

78. The apparatus of claim 76, wherein the ranging results comprise laser ranging results, and wherein the processor filters one or more ranging results corresponding to each detection frame information, comprising:

79. The apparatus as claimed in claim 78, wherein the processor determines validity of each of the ranging results according to the detection frame information corresponding to each of the ranging results and the laser spot corresponding to each of the ranging results, comprising:

80. The apparatus as claimed in claim 79 wherein said processor determines validity of each of said ranging results based on said area coincidence ratio, comprising:

comparing the area coincidence rate with a preset proportion threshold;

81. The apparatus as claimed in claim 78, wherein in the case that it is determined that each of the detection frame information corresponds to a plurality of ranging results, the processor determines validity of each of the ranging results according to the detection frame information corresponding to each of the ranging results and the laser spot corresponding to each of the ranging results, comprising:

82. The apparatus of claim 75, wherein the processor determines the status information of the target object according to the plurality of detection frame information and the valid ranging result, comprising:

83. The apparatus of claim 82, wherein the processor determines valid ranging results for each of the detection block information, comprising:

84. The apparatus of claim 82, wherein in the case that each of the detection frame information corresponds to a plurality of valid ranging results, the processor determines the state information of the target object according to each of the detection frame information and the valid ranging result corresponding to each of the detection frame information, comprising:

85. The apparatus of claim 75, wherein the processor further performs the following:

86. The apparatus as claimed in claim 85, wherein the processor filters the valid ranging result corresponding to each of the detection frame information according to the physical estimated size of the target object corresponding to each of the detection frame information, comprising:

87. The apparatus according to claim 86, wherein the processor determines a physical estimated size of the target object for each of the detection box information, comprising:

88. The apparatus according to claim 86, wherein the processor further performs the following:

89. The apparatus according to claim 86, wherein the processor further performs the following:

and after filtering the effective ranging result corresponding to the detection frame information, determining the effective ranging result corresponding to the detection frame information adjacent to the sampling time of the detection frame information with the effective ranging result filtered as the effective ranging result corresponding to the detection frame information with the effective ranging result filtered.

90. The apparatus of claim 75, wherein the processor determines the status information of the target object according to the plurality of detection frame information and the valid ranging result, comprising:

91. The apparatus according to claim 90, wherein the status information comprises location information; the determining, by the processor, the state information of the target object according to each piece of target detection frame information and the effective ranging result corresponding to each piece of target detection frame information includes:

determining initial position information related to the target object corresponding to each target detection frame information according to each target detection frame information and an effective ranging result corresponding to each target detection frame information to obtain a plurality of initial position information; and

screening the initial position information according to the sampling time point corresponding to each target detection frame information to obtain one or more effective initial position information;

92. The apparatus according to claim 91, wherein the processor filters the plurality of initial position information according to the sampling time point corresponding to each target detection frame information, including:

93. The apparatus of claim 92, wherein the processor sequentially filters initial position information about the target object corresponding to each of the target detection frame information in a time sequence, comprising:

comparing the time difference to a state variable threshold;

94. The apparatus of claim 93, wherein the state variable threshold varies with determined valid initial position information.

95. The apparatus of claim 94, wherein the state variable threshold is a time difference between the current initial position information being filtered and a sampling time point of valid initial position information adjacent to the sampling time of the current initial position information being filtered.

96. The apparatus as recited in claim 90, wherein said processor further performs the following:

97. The apparatus according to claim 96, wherein the processor determines whether the target object has moved according to a predetermined condition based on a plurality of the detection frame information, including:

98. The apparatus according to claim 97 wherein the processor determines whether the target object has moved according to a predetermined condition based on the first probability distribution and the second probability distribution, including:

99. The apparatus of claim 91, wherein the processor determines the state information of the target object based on one or more of the valid initial position information, comprising:

100. The apparatus of claim 99, wherein the processor optimizes one or more of the valid initial position information comprising:

101. The apparatus of claim 100, wherein the target deviation comprises a first deviation and/or a second deviation;

102. The apparatus of claim 101, wherein the first deviation is characterized by a probability density function of target detection frame information and valid ranging results used for calculating the valid initial position information.

103. The apparatus of claim 101, wherein the processor optimizes one or more of the valid initial position information, further comprising:

and filtering the abnormal optimized position information.

104. The apparatus according to claim 103, wherein the target deviation comprises the first deviation, the first deviation comprises a first sub-deviation and/or a second sub-deviation;

wherein the determining, by the processor, whether the optimized position information corresponding to each effective initial position information after the nonlinear optimization is abnormal includes:

105. The apparatus of claim 103, wherein the processor optimizes one or more of the valid initial position information, further comprising:

106. The apparatus of claim 99, wherein the processor further performs the following:

107. The apparatus according to claim 106, wherein the processor predicts the state information of the target object based on the position information of the target object when lost and the smoothed motion trajectory of the target object comprises:

108. The apparatus of claim 107, wherein a spatial occupancy of the probability distribution of the predicted location increases as a loss time of the target object increases.

109. The apparatus according to claim 108, wherein the parameter of the variation of the spatial aspect of the probability distribution of predicted positions is related to the type of the target object.

110. The apparatus according to claim 109, wherein the variation parameter comprises a rate of increase of a spatial proportion of the probability distribution of the predicted location;

111. The apparatus according to claim 110, wherein the first rate of increase is less than the second rate of increase.

112. An apparatus for determining status information of a target object, comprising:

a processor;

a readable storage medium for storing one or more programs,

113. The apparatus of claim 112, wherein the processor determines the status information of the target object according to one or more target detection frame information and a plurality of ranging results, comprising:

114. The apparatus according to claim 113, wherein the status information comprises location information; wherein the processor determines one or more target detection frame information satisfying a preset condition from among the plurality of detection frame information, and includes:

115. The apparatus according to claim 114, wherein the processor filters the plurality of detection frame information according to the sampling time point corresponding to each detection frame information, including:

116. The apparatus as claimed in claim 115, wherein the processor sequentially filters each of the detection frame information in time order comprises:

comparing the time difference to a state variable threshold;

117. The apparatus according to claim 116, wherein the state variable threshold varies with determined target detection box information.

118. The apparatus according to claim 117, wherein the state variable threshold is a time difference between sampling time points of the currently-being-screened test frame information and target test frame information adjacent to the sampling time of the currently-being-screened test frame information.

119. The apparatus according to claim 112, wherein the processor determines a plurality of target detection box information satisfying a predetermined condition from among the plurality of detection box information, comprising:

120. The apparatus as claimed in claim 119, wherein said processor determines whether the target object has moved according to a predetermined condition based on a plurality of the detection frame information, comprising:

121. The apparatus according to claim 120 wherein the processor determines whether the target object has moved according to the first probability distribution and the second probability distribution, the movement satisfying a predetermined condition comprising:

122. The apparatus as claimed in claim 114, wherein said processor determines the status information of the target object according to one or more target detection frame information and a plurality of the ranging results, comprising:

123. The apparatus of claim 122, wherein the processor optimizes valid initial position information corresponding to one or more of the target detection frame information, comprising:

124. The apparatus according to claim 123, wherein the target deviation comprises a first deviation and/or a second deviation;

125. The apparatus of claim 124, wherein the first deviation is characterized by a probability density function of target detection frame information and ranging results used to calculate the valid initial position information.

126. The apparatus as claimed in claim 124 wherein the processor optimizes valid initial position information corresponding to one or more of the target detection frame information, further comprising:

and filtering the abnormal optimized position information.

127. The apparatus according to claim 126, wherein the target deviation comprises the first deviation, the first deviation comprises a first sub-deviation and/or a second sub-deviation;

128. The apparatus of claim 126, wherein the processor optimizes valid initial position information corresponding to one or more of the target detection frame information, further comprising:

129. The apparatus as claimed in claim 120 wherein said processor further performs the following:

130. The apparatus according to claim 129 wherein the processor predicts the state information of the target object based on the position information of the target object when lost and the smoothed motion trajectory for the target object, comprising:

131. The apparatus of claim 130, wherein a spatial fraction of the probability distribution of the predicted location increases as a time of loss of the target object increases.

132. The apparatus according to claim 131, wherein a parameter of variation of the spatial aspect of the probability distribution of predicted positions is related to the type of the target object.

133. The apparatus according to claim 132, wherein the variation parameter comprises a speed of increase of a spatial proportion of the probability distribution of the predicted location, and wherein the correlation of the variation parameter of the spatial proportion of the probability distribution of the predicted location with the type of the target object comprises:

134. The apparatus according to claim 133, wherein the first rate of growth is less than the second rate of growth.

135. The apparatus of claim 112, wherein the processor determines the status information of the target object according to one or more target detection frame information and a plurality of ranging results, comprising:

136. The apparatus as claimed in claim 135, wherein said processor screens out valid ranging results from a plurality of said ranging results according to one or more said target detection frame information, comprising:

137. The apparatus of claim 136, wherein the processor determines one or more ranging results corresponding to each target detection block information, comprising:

138. The apparatus as claimed in claim 136, wherein the ranging results comprise laser ranging results, and the processor performs a filtering on one or more ranging results corresponding to each target detection frame information, comprising:

139. The apparatus as claimed in claim 138, wherein the processor determines validity of each of the ranging results according to the target detection frame information corresponding to each of the ranging results and the laser spot corresponding to each of the ranging results, comprising:

140. The apparatus as claimed in claim 139, wherein said processor determining validity of each of said ranging results based on said area coincidence rates comprises:

comparing the area coincidence rate with a preset proportion threshold;

141. The apparatus as claimed in claim 138, wherein in the case that it is determined that each target detection frame information corresponds to a plurality of ranging results, the processor determines validity of each ranging result according to the target detection frame information corresponding to each ranging result and the laser spot corresponding to each ranging result, comprising:

142. The apparatus as claimed in claim 135, wherein said processor determining the status information of the target object according to the plurality of target detection frame information and the valid ranging result comprises:

143. The apparatus as claimed in claim 142, wherein the processor determines valid ranging result corresponding to each target detection block information, comprising:

144. The apparatus as claimed in claim 142, wherein in the case that each of the target detection frame information corresponds to a plurality of valid ranging results, the processor determines the status information of the target object according to each of the target detection frame information and the valid ranging result corresponding to each of the target detection frame information, comprising:

145. The apparatus of claim 112, wherein the processor determines the status information of the target object according to one or more target detection frame information and a plurality of ranging results, comprising:

146. The apparatus as claimed in claim 145, wherein the processor filters the ranging result corresponding to each target detection frame information according to the physical estimated size of the target object corresponding to each target detection frame information, comprising:

147. The apparatus according to claim 146, wherein the processor determines a physical estimated size of the target object for each of the target detection box information, comprising:

148. The apparatus as claimed in claim 146 and wherein said processor further performs the operations of:

149. A system for determining status information of a target object, comprising:

an imaging device for obtaining a plurality of frame images about the target object;

a status information determination apparatus as claimed in any one of claims 75 to 148.

150. A movable platform, comprising:

a movable body; and

the system of claim 149.

151. A readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 74.