CN115205324A - Target object orientation determining method and device - Google Patents

Abstract

The embodiment of the specification provides a target object orientation determining method and a target object orientation determining device, wherein the target object orientation determining method comprises the steps of receiving an i +1 th video frame containing a target object, and acquiring the shape, key points and an edge detection frame of the target object in the i +1 th video frame; determining a target attribute value of the target object based on the shape of the target object and the keypoints; determining a first orientation of a target object in the (i + 1) th video frame according to the target attribute value of the target object and an edge detection frame; determining a second orientation of the target object in the i +1 th video frame based on the target orientation of the target object in the i +1 th video frame, and determining the target orientation of the target object in the i +1 th video frame based on the first orientation and the second orientation.

Description

Target object orientation determination method and device

technical field

The embodiments of the present specification relate to the field of computer technologies, and in particular, to a method for determining the orientation of a target object. One or more embodiments of the present specification simultaneously relate to an apparatus for determining the orientation of a target object, a computing device, and a computer-readable storage medium.

Background technique

Under the background of the gradual maturity of vehicle autonomous driving, the pursuit of landing, and the mass production, the vehicle's perception of obstacles and structured output also have higher requirements for stability, comprehensiveness, and accuracy. In the field of automatic driving, the calculation of the orientation of the vehicle is a basic task of the perception system, which is a part of the pose estimation of the vehicle. The subsequent trajectory prediction and planning control of the vehicle also depend on the orientation calculation. The orientation calculation of the vehicle is mainly based on lidar, but for scenes without lidar or in the blind spot of lidar, the direction of the vehicle cannot be well identified.

Therefore, there is an urgent need to provide a method for determining the orientation of a target object that can improve the accuracy and stability of vehicle orientation recognition.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present specification provide a method for determining the orientation of a target object. One or more embodiments of the present specification simultaneously relate to an apparatus for determining the orientation of a target object, a computing device, and a computer-readable storage medium, so as to solve the technical defects existing in the prior art.

According to a first aspect of the embodiments of the present specification, a method for determining the orientation of a target object is provided, including:

Receive the i+1 th video frame including the target object, and obtain the shape, key points and edge detection frame of the target object in the i+1 th video frame;

Determine the target attribute value of the target object based on the shape and key points of the target object;

Determine the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame;

The second orientation of the target object in the i+1 th video frame is determined based on the target orientation of the target object in the ith video frame, and the i+1 th video frame is determined based on the first orientation and the second orientation The target orientation of the target object in .

According to a second aspect of the embodiments of the present specification, there is provided an apparatus for determining the orientation of a target object, including:

The first video receiving module is configured to receive the i+1 th video frame including the target object, and obtain the shape, key points and edge detection frame of the target object in the i+1 th video frame;

a first determining module, configured to determine a target attribute value of the target object based on the shape and key points of the target object;

a second determining module, configured to determine the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame;

a first target orientation determination module configured to determine a second orientation of the target object in the i+1 th video frame based on the target orientation of the target object in the i th video frame, based on the first orientation and the second orientation Determine the target orientation of the target object in the i+1 th video frame.

According to a third aspect of the embodiments of the present specification, a computing device is provided, including:

memory and processor;

The memory is used for storing computer-executable instructions, the processor is used for executing the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method for determining the orientation of the target object are implemented.

According to a fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, implements the steps of the method for determining the orientation of a target object.

An embodiment of the present specification implements a method and apparatus for determining the orientation of a target object, wherein the method for determining the orientation of a target object includes receiving the i+1 th video frame including the target object, and acquiring the i+1 th video frame the shape, key points and edge detection frame of the target object in the frame; determine the target attribute value of the target object based on the shape and key points of the target object; determine the target attribute value of the target object and the edge detection frame The first orientation of the target object in the i+1 th video frame; the second orientation of the target object in the i+1 th video frame is determined based on the target orientation of the target object in the i th video frame, based on the first orientation and The second orientation determines the target orientation of the target object in the i+1th video frame; specifically, the target object orientation determination method decomposes the calculation of the target orientation into shape estimation and target attribute values of the target object. Multiple steps such as calculation and orientation calculation are implemented, so that each implementation step can be well decoupled and integrated, and finally an accurate and stable target orientation of the target object can be obtained.

Description of drawings

FIG. 1 is an example diagram of a specific application scenario of a method for determining the orientation of a target object provided by an embodiment of the present specification;

2 is a flowchart of a method for determining the orientation of a target object provided by an embodiment of the present specification;

3 is a schematic diagram of a video frame including a target object in a method for determining the orientation of a target object provided by an embodiment of the present specification;

4 is a schematic diagram of the projection relationship of the target object in the world coordinate system in a method for determining the orientation of a target object provided by an embodiment of the present specification;

FIG. 5 is a flowchart of an application of a method for determining the orientation of a target object provided in an embodiment of the present specification to automatic driving of a vehicle;

6 is a schematic structural diagram of an apparatus for determining the orientation of a target object provided by an embodiment of the present specification;

FIG. 7 is a structural block diagram of a computing device provided by an embodiment of the present specification.

Detailed ways

In the following description, numerous specific details are set forth in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar promotions without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementation disclosed below.

The terminology used in one or more embodiments of this specification is for the purpose of describing a particular embodiment only and is not intended to limit the one or more embodiments of this specification. As used in the specification or embodiments and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used in this specification in one or more embodiments refers to and includes any and all possible combinations of one or more of the associated listed items.

It will be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, a first could be termed a second, and similarly, a second could be termed a first, without departing from the scope of one or more embodiments of this specification. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."

First, the terminology involved in one or more embodiments of the present specification is explained.

Projective geometry: Different from Euclidean geometry, usually used in real-world and image plane transformations and calculations.

Long tail problem: Limited by the richness of samples and the performance of models, the field of autonomous driving usually faces many corner cases (CornerCases) that make existing systems make mistakes.

Interpretability: An end-to-end model, usually a black box, suffers from weak interpretability.

Edge computing: It is required to complete real-time computing on the end, usually emphasizing high real-time performance, which is different from cloud computing.

Camera extrinsics: Camera extrinsics, camera extrinsics are parameters in the world coordinate system, such as camera position, rotation direction, etc.

Kalman filter: Kalman filter is an algorithm that uses the linear system state equation to optimally estimate the system state through the system input and output observation data.

Bicycle model: that is, a kinematic automatic car model. When the current vehicle orientation is obtained, the bicycle model can be used to predict the vehicle orientation at the next moment.

In the field of vehicle autonomous driving, the calculation of the orientation of the vehicle is a basic task of a perception system, which is a part of the pose estimation of the vehicle. The vehicle trajectory prediction and planning control in the subsequent process also depend on the orientation. The embodiment of this specification provides a method for determining the orientation of a target object based on vision, so as to realize more accurate calculation of the orientation of the vehicle. For the application of the vehicle equipped with the lidar configuration, the high-precision visual orientation calculation method of this specification is combined with the application. , which can greatly improve the accuracy and stability of orientation calculation; for vehicle application scenarios without lidar or in the radar blind area, the vision solution provided in this manual can independently undertake orientation calculation and output, so that the target object provided in this manual is oriented towards The method can be applied to various application scenarios to improve user experience.

Based on this, in this specification, a method for determining the orientation of a target object is provided. One or more embodiments of this specification simultaneously relate to an apparatus for determining the orientation of a target object, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.

Referring to FIG. 1 , FIG. 1 shows an example diagram of a specific application scenario of a method for determining the orientation of a target object provided by an embodiment of the present specification.

The application scenario of FIG. 1 includes an image capture terminal 102 , an image receiving terminal 104 and a server 106 . Specifically, the image receiving terminal 104 receives the image a including the vehicle captured by the image capture terminal 102 in real time; after the image receiving terminal 104 receives the image a , send the image a to the server 106, after the server 106 receives the image a, input the image a into the vehicle type detection model to obtain the vehicle type of the vehicle in the image a; input the image a into the key point detection model to obtain the vehicle type in the image a key points; and input the image a into the whole-vehicle detection model to obtain the whole-vehicle detection frame of the vehicle in the image a; among them, the vehicle-type detection model, the key point detection model and the whole-vehicle detection model can be understood as using convolutional neural network training. Deep learning models.

In the specific calculation, first use the model of the vehicle in the image a to determine the range of the length and width of the vehicle, that is, the length and width of the vehicle is 1; for example, the model of the vehicle is an SUV. It is about 1.8m, the length of a large SUV is about 4.7m~5m, and the width is about 1.9m, then the length range of the vehicle determined based on the model of the vehicle is 4.4m~5m, and the width range of the vehicle is 1.8m~1.9m .

Then use the key points of the vehicle in the image a and the external parameters of the camera that obtained the image a to calculate the real length and width of the vehicle in the image a, that is, the vehicle length and width 2; where the key points are in the image a in FIG. 1 . E, F, D, G points, etc. If the image a is the first frame image including the vehicle, then if the vehicle length and width 2 is within the range of the vehicle length and width 1, the vehicle length and width 2 is taken as the target length and width of the vehicle in the image a, that is, The length and width of the vehicle in Fig. 1 are 3; if the image a contains other second, third or fourth frame images after the first frame image of the vehicle, the vehicle's The real length and width predict the predicted length and width of the vehicle in the current image a, and then predict the actual length and width of the vehicle in the image a obtained through the key points of the vehicle and the camera external parameters and the length and width of the vehicle in the previous frame image. The predicted length and width of the vehicle in the image a is input into the Kalman filter for fusion correction to obtain the target length and width of the vehicle in the image a. In practical applications, the camera extrinsic parameters corresponding to each frame of image will also change during the driving process of the vehicle. Therefore, when calculating the target length and width of the vehicle in the image, it is necessary to iteratively calculate the length and width of the vehicle based on the camera extrinsic parameters. In order to obtain the accurate length and width of the vehicle in each frame of image, to ensure the accuracy of the subsequent calculation of the vehicle orientation.

After obtaining the target length and width of the vehicle in the image a, that is, the vehicle length and width 3, the entire vehicle detection frame of the vehicle in the image a is projected in the preset coordinate system through projective geometry, and the entire vehicle in the image a is obtained. The mapping key points of the vehicle detection frame in the preset coordinate system, where the preset coordinate system is the world coordinate system of the camera. After the target length and width of the vehicle in the image a and the mapping key points are determined, the image The target length and width of the vehicle in a and the coordinate values of the mapping key points in the world coordinate system are calculated to obtain the vehicle orientation of the vehicle in the image a. In practical applications, the image a is the first frame image containing the vehicle, then the vehicle The orientation is the target vehicle orientation of the vehicle in the image a; and if the image a has an image containing the vehicle in the previous frame, the predicted vehicle orientation of the vehicle in the image a can be obtained through the single-vehicle model, and then the image The predicted vehicle orientation of the vehicle in a and the vehicle orientation calculated by the length and width of the vehicle and the mapping key points are input into the Kalman filter for processing, and the target vehicle orientation of the vehicle in the image a is obtained.

The method for determining the orientation of the target object provided in the embodiments of this specification is applied to the calculation of the orientation of the vehicle, and the calculation of the orientation of the vehicle is decomposed into multiple parts such as model estimation, length and width calculation, and Kalman filtering, which can be decoupled through different implementations. and fusion, and finally obtain a more accurate and stable orientation of the vehicle, and the overall orientation calculation model of the vehicle only relies on key points, and the amount of calculation itself is small. It has great advantages in accuracy, stability, scope of application and real-time performance.

Referring to FIG. 2, FIG. 2 shows a flowchart of a method for determining the orientation of a target object provided according to an embodiment of the present specification, which specifically includes the following steps.

Step 202: Receive the i+1 th video frame containing the target object, and acquire the shape, key points and edge detection frame of the target object in the i+1 th video frame.

Among them, the target objects include but are not limited to two-wheeled vehicles, three-wheeled vehicles, four-wheeled vehicles or other multi-wheeled vehicles, etc., or logistics vehicles, public service vehicles, medical service vehicles, terminal service vehicles, etc.; in addition, i is a positive Integer, for example, if i is 1, then i+1 is 2. In practical applications, the method for determining the orientation of the target object provided in the embodiments of this specification can predict the orientation of a stationary vehicle, or can also perform a calculation on a moving vehicle. Prediction, in order to facilitate understanding, this specification takes a four-wheeled vehicle in motion as an example to introduce the solution in detail.

Specifically, receiving the i+1 th video frame containing the target object can be understood as receiving the i+1 th video frame obtained by the camera and containing the moving vehicle, and obtaining the i+1 th video frame in the video frame. The shape, keypoints, and edge detection boxes of the target object.

Taking i as 1 as an example, then the i+1th video frame is the second video frame containing the moving vehicle; after obtaining the second video frame, obtain the shape and key of the vehicle in the second video frame. Point and edge detection frame; among them, the shape of the vehicle can be understood as the model of the vehicle, and the edge detection frame of the vehicle can be understood as the whole vehicle detection frame of the vehicle.

During specific implementation, the acquiring the shape, key points and edge detection frame of the target object in the i+1 th video frame includes:

The i+1 th video frame is input into the first recognition model, the second recognition model and the third recognition model respectively, and the shape, key points and edge detection frame of the target object in the i+1 th video frame are acquired.

The first recognition model, the second recognition model and the third recognition model may be understood as deep learning models trained by using a convolutional neural network.

Following the above example, input the second video frame into the first recognition model, the vehicle type in the second video frame can be obtained; input the second video frame into the second recognition model, the vehicle type in the second video frame can be obtained. Key points; input the second video frame into the third recognition model, the whole-vehicle detection frame of the vehicle in the second video frame can be obtained; the whole-vehicle detection frame of the vehicle can show the range of the 3D detection of the vehicle and the edge information of the vehicle; The key points of the vehicle can show wheel points and light points, with clear physical meaning and texture, and the key point information of the vehicle also includes the information of the side frame of the vehicle.

In the embodiment of this specification, the shape, key points and edge detection frame of the target object are obtained through different recognition models, and a targeted model is used to accurately obtain each feature of the target object, and then the fusion calculation is performed later. , to obtain the accurate orientation of the target object. Compared with the original large model, it is faster and more accurate to directly obtain the orientation of the target object. For example, the key point model is used to obtain the key points of the target object. The key point model has strong image texture. Compared with the original large model, the analysis ability has great advantages in the stability of the model and the handling of the long tail problem.

Referring to FIG. 3, FIG. 3 shows a schematic diagram of a video frame including a target object in a method for determining the orientation of a target object provided according to an embodiment of the present specification.

As can be seen from Figure 3, the target object in the video frame is a vehicle, and the video frames containing the vehicle in Figure 3 are input into the three recognition models respectively to obtain the whole vehicle detection frame of the vehicle in The rectangular edge detection frame composed of ABCD around the vehicle is used to obtain the key points of the vehicle in Figure 3: E, F, G, and D, and the model of the vehicle in Figure 3 is obtained.

Specifically, before the receiving the i+1 th video frame including the target object, the method further includes:

Receive the ith video frame containing the target object, and obtain the shape, key points and edge detection frame of the target object in the ith video frame;

Determine the target attribute value of the target object based on the shape and key points of the target object;

The target orientation of the target object in the ith video frame is determined according to the target attribute value of the target object and the edge detection frame.

Wherein, the target object in the i-th video frame and the target object in the i+1-th video frame are the same target object.

In practical applications, the method for determining the orientation of the target object is applied to the scene where the vehicle is driving. For a moving vehicle, when calculating the orientation of the vehicle, in order to obtain an accurate and stable orientation of the vehicle, the current frame of the vehicle will be used. The orientation of the vehicle at the previous moment is predicted to the orientation of the vehicle in the current frame, and then the target orientation of the vehicle in the current frame is obtained based on the predicted vehicle orientation and the vehicle orientation calculated through the length and width of the vehicle in the current frame and key points.

Therefore, when determining the target orientation of the vehicle in the i+1 th video frame, it is necessary to obtain the target orientation of the vehicle in the i th video frame.

Following the above example, if i is still 1, first receive the first video frame containing the vehicle, and obtain the shape, key points and edge detection frame of the vehicle in the first video frame; then determine the vehicle based on the shape and key points of the vehicle. Finally, the target orientation of the target object in the first video frame is determined according to the target attribute value of the vehicle and the edge detection frame.

Wherein, for the method of obtaining the shape, key points and edge detection frame of the target object in the i-th video frame, reference may be made to the shape, key points and edge detection frame of the target object in the i+1-th video frame in the above-mentioned embodiment. The specific introduction will not be repeated here.

During specific implementation, the determining the target attribute value of the target object based on the shape and key points of the target object includes:

Determine the first initial attribute value of the target object in the ith video frame based on the shape of the target object in the ith video frame;

Determine the second initial attribute value of the target object in the i-th video frame based on the key points of the target object in the i-th video frame and the camera extrinsic parameters for obtaining the i-th video frame;

In the case that the second initial attribute value is less than or equal to the first initial attribute value, the second initial attribute value is used as the target attribute value of the target object in the ith video frame.

Among them, the attribute value can be understood as the length and width, then when the target object is a vehicle, the attribute value can be understood as the length and width of the vehicle.

Specifically, in the case where the i-th video frame is the first video frame containing the target object, and there are no other video frames before the first video frame, the key points of the target object in the i-th video frame and the For the camera extrinsic parameters of the i-th video frame, the second initial attribute value of the target object in the i-th video frame obtained by calculation is used as the target attribute value, and the target attribute value must also be within the first initial attribute value.

Following the above example, after obtaining the vehicle type, key points and the whole vehicle detection frame in the first video frame, first determine the first initial length and width of the vehicle according to the vehicle type, that is, the length and width range of the vehicle; The key points of the vehicle in the first video frame and the extrinsic parameters of the camera that acquired the video frame are calculated to obtain the second initial length and width of the vehicle in the first video frame; finally, the second initial length and width are less than or equal to the first initial length and width In the case of , take the second initial length and width as the target length and width of the vehicle in the first video frame.

In practical applications, the maximum length and width and the minimum length and width of the model can be obtained according to the model of the vehicle, and then the maximum length and width and the minimum length and width of the model are used as the range of the length and width of the vehicle. After the camera extrinsic parameters are calculated to obtain the real length and width of the vehicle, if the real length and width of the vehicle are within the range of the length and width, it means that the calculation of the real length and width of the vehicle according to the key points of the vehicle and the camera extrinsic parameters is accurate, and can be used as The target length and width of the vehicle in the first video frame; if the real length and width of the vehicle is not within the range of length and width, it means that the calculation of the real length and width of the vehicle based on the key points of the vehicle and the external parameters of the camera is wrong.

In the embodiment of this specification, in the case where the i-th video frame is the first video frame containing the target object, the first initial attribute value determined by the shape of the target object is used as the constraint condition, so that the key points of the target object and The second initial attribute value of the vehicle in the ith video frame calculated by the camera extrinsic parameters of the ith video frame is obtained as the target attribute value, and the orientation of the target object in the first video frame can be quickly obtained subsequently.

Step 204: Determine a target attribute value of the target object based on the shape and key points of the target object.

Specifically, determining the target attribute value of the target object based on the shape and key points of the target object includes:

Based on the shape of the target object in the i+1 th video frame, determine the first initial attribute value of the target object in the i+1 th video frame;

Determine the second initial attribute of the target object in the i+1 th video frame based on the key points of the target object in the i+1 th video frame and the camera extrinsic parameters used to obtain the i+1 th video frame value;

In the case that the second initial attribute value is less than or equal to the first initial attribute value, determine the target object in the i+1 th video frame based on the target attribute value of the target object in the i th video frame The third initial attribute value of ;

The target attribute value of the target object in the i+1 th video frame is determined based on the second initial attribute value and the third initial attribute value.

Specifically, the specific calculation method of the target attribute value of the target object in the i+1 th video frame is different from the calculation method of the target attribute value of the target object in the ith video frame in the above embodiment.

First, determine the first initial attribute value of the target object in the i+1th video frame based on the shape of the target object in the i+1th video frame, and then obtain the i+1th video frame according to the key points of the target object in the i+1th video frame and obtain the i +1 camera external parameters of the target object in the video frame, obtain the second initial attribute value of the target object in the i+1th video frame; for the first initial attribute value of the target object in the i+1th video frame and the The calculation of the second initial attribute value is the same as the above-mentioned calculation method for the first initial attribute value and the second initial attribute value of the target object in the ith video frame, and details are not described herein.

Since the first initial attribute value of the target object in the i+1 th video frame is a constraint condition of the second initial attribute value, then the first initial attribute value and the second initial attribute value of the target object in the i+1 th video frame are obtained. After the attribute value, it is necessary to judge whether the second initial attribute value of the target object in the i+1th video frame is within the first initial attribute value, if so, it can be based on the target attribute value of the target object in the ith video frame, Determine the third initial attribute value of the target object in the i+1 th video frame; in practical applications, the target attribute value of the target object in the ith video frame is input into the Kalman filter to predict and obtain in the i+1 th video frame. The third initial property value of the target object.

Finally, the second initial attribute value and the third initial attribute value of the target object in the i+1 th video frame are fused and corrected to obtain the target attribute value of the target object in the i+1 th video frame.

The method for determining the orientation of the target object in the embodiment of this specification is applied in a vehicle driving scene. The camera will acquire an image containing the vehicle in real time and send it to the server. After the server receives the image containing the vehicle, it will be based on the current video frame. The length and width of the vehicle in the video frame predict the length and width of the vehicle in the current video frame, and then the predicted length and width of the vehicle and the length and width of the vehicle obtained through the key points of the vehicle in the current frame and the camera extrinsic parameters of the current video frame are obtained. The width is fed into the Kalman filter for fusion correction to obtain an accurate and stable vehicle length and width of the vehicle in the current video frame.

Step 206: Determine the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame.

Specifically, the determining the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame includes:

Mapping the edge detection frame of the target object in the i+1 th video frame in a preset coordinate system to obtain the mapping key point of the edge detection frame of the target object in the i+1 th video frame;

The first orientation of the target object in the i+1 th video frame is determined based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object.

Among them, the preset coordinate system can be understood as the world coordinate system of the camera. In practical applications, when calculating the first orientation of the target object in the i+1th video frame, the target object in the i+1th video frame is firstly calculated. The edge detection frame of the object is mapped in the world coordinate system, and the mapping key point of the edge detection frame of the target object in the i+1 video frame is obtained; then based on the coordinate value of the mapping key point in the world coordinate system and the i+ The target attribute value of the target object in one video frame is calculated to obtain the first orientation of the target object in the i+1th video frame.

Referring to FIG. 4 , FIG. 4 shows a schematic diagram of the projection relationship of the target object in the world coordinate system in the method for determining the orientation of the target object provided according to an embodiment of the present specification.

Combined with Figure 3, the rectangle formed by ABCD in Figure 3 is the edge detection frame of the vehicle, and ABC in Figure 4 is the three visible points of the vehicle. Through the camera imaging, the ABC of the vehicle is in the lowermost horizontal line in Figure 4. The corresponding three points on the line (ie the imaging plane) are A', B', C', and according to the XOY coordinate system in Figure 4, the specific pixel values of A', B', C' on the imaging plane can be obtained, That is, the coordinate value, according to the coordinate values of A', B', C' on the imaging plane and AC (the width of the vehicle), BC (the length of the vehicle), the direction of BC in the XOY coordinate system can be obtained, which is the vehicle. The orientation in the video frame.

In the embodiment of this specification, the geometric modeling method is used to calculate the orientation of the target object. The principle design is not sensitive to visual imaging, and it is suitable for pinhole and fisheye cameras at the same time, so it is not sensitive to modules with different parameters. It has good mass production characteristics; and using the constraint relationship between the length, width and orientation of the target object, the coordinates of the key points in the world coordinate system and the length and width of the target object can be calculated through the mapping of the edge detection frame of the target object. Get a more accurate orientation of the target object in the video frame.

Specifically, the determining the first orientation of the target object in the i+1 th video frame based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object, including :

Determine the first target edge and the second target edge of the target object based on the edge detection frame;

Determine the edge value of the first target edge and the edge value of the second target edge according to the target attribute value of the target object;

Based on the coordinate value of the mapping key point in the preset coordinate system, the edge value of the first target edge and the edge value of the second target edge, calculate and obtain the i+1 th video frame. The first orientation of the target object.

Among them, the first target edge is the long side of the target object, and the second target edge is the wide side of the target object. In practical applications, based on the edge detection frame of the target object, it can be determined which side is the length of the target object and which side is the target object. After the long side and wide side of the target object are determined, according to the target attribute value of the target object, length information is given to the long side, width information is given to the wide side, and then the coordinates of the key points in the world coordinate system are mapped. The value, the length of the long side of the target object and the width of the wide side are calculated to obtain the orientation of the target object.

In the embodiment of this specification, using the constraint relationship between the length, width and orientation of the target object, the coordinates of the key points of the edge detection frame of the target object in the world coordinate system and the length and width of the target object can be calculated to obtain relatively accurate The orientation of the target object in the video frame of .

Step 208: Determine the second orientation of the target object in the i+1 th video frame based on the target orientation of the target object in the ith video frame, and determine the i+1 th orientation based on the first orientation and the second orientation The target orientation of the target object in each video frame.

Specifically, determining the target orientation of the target object in the ith video frame according to the target attribute value of the target object and the edge detection frame includes:

The edge detection frame of the target object in the i-th video frame is mapped in a preset coordinate system, and the mapping key point of the edge detection frame of the target object in the i-th video frame is obtained;

Determine the first orientation of the target object in the i-th video frame based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object;

In the case where i is 1, the first orientation of the target object in the i-th video frame is used as the target orientation of the target object in the i-th video frame.

Specifically, in the case where the i-th video frame is the first video frame containing the target object, in the case of calculating the target orientation of the target object in the i-th video frame, directly map key points in the world coordinate system The target orientation of the i-th video frame can be obtained by calculating the coordinate value of , and the length and width of the target object; and if the i-th video frame is not the first video frame, you can refer to the i+1-th video frame. The acquisition method of the target orientation of the target object in the video frame realizes the acquisition of the target orientation.

In the embodiment of this specification, if the i-th video frame is the first video frame containing the target object or a still video frame, there is no constraint relationship with speed, distance, etc., then directly through the mapping key The target orientation of the ith video frame can be quickly obtained by calculating the coordinate value of the point in the world coordinate system and the length and width of the target object.

Wherein, determining the first orientation of the target object in the ith video frame based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object includes:

Determine the first target edge and the second target edge of the target object based on the edge detection frame;

Determine the edge value of the first target edge and the edge value of the second target edge according to the target attribute value of the target object;

Based on the coordinate value of the mapping key point in the preset coordinate system, the edge value of the first target edge and the edge value of the second target edge, calculate and obtain the target object in the i-th video frame the first orientation.

In the embodiment of this specification, in the case where the i-th video frame is the first video frame or a still video frame, the mapping key points of the edge detection frame of the target object in the i-th video frame and the length of the target object may be used. The first orientation of the target object in the ith video frame is obtained by calculating the width and width, and a more accurate orientation of the target object is obtained by using the constraint relationship between the length, width and orientation of the target object.

However, when the video frame is not the first video frame and the video frame is not a still video frame, in order to obtain a stable and accurate target orientation of the target object, the orientation of the target object needs to be updated through the bicycle model and filter. Good calculation and correction, the specific implementation is as follows:

The determining of the second orientation of the target object in the i+1 th video frame based on the target orientation of the target object in the ith video frame includes:

Input the target orientation of the target object in the i-th video frame into the bicycle model, and obtain the second orientation of the target object in the i+1-th video frame.

In practical applications, a bicycle model of the vehicle is pre-established, and the above calculated orientation is used as the observation of the Kalman filter. In the bicycle model, the direction of the vehicle is consistent with the direction of the speed, and the difference of the distance measurement in the time series provides the speed information. Therefore, for a vehicle with a large moving speed, the direction of the speed can be more trusted as the orientation, and for a vehicle with a slow and stationary motion, the orientation calculated by the geometric constraints can be more trusted. For a moving vehicle, the motion speed of the vehicle can be obtained through the time-series ranging or lidar of the vehicle, and the target orientation of the target object in the i-th video frame is input into the bicycle model. The constraint relationship of the motion speed can predict the second orientation of the target object in the i+1th video frame.

Then input the first and second orientations of the target object in the i+1th video frame into the Kalman filter, and the Kalman filter performs weighted fusion of the first and second orientations to obtain the i+1th video frame The target direction of the target object in to the effect.

In the embodiment of this specification, the method for determining the orientation of the target object proposes a new geometric modeling solution, which is suitable for near and far vehicles, as well as stationary and moving vehicles, without the limitation of principle, and through geometric modeling The model replaces the deep learning model in the existing technology, which has advantages in terms of computing resources and time-consuming, and can be suitable for automatic driving applications with strong real-time performance; and this solution disassembles the end-to-end deep learning model into multiple sub-models and new geometric modeling, which increases redundancy, has advantages in stability and reduces long-tail problems, and improves the safety of autonomous driving applications; while the corresponding geometric modeling, the principle design is not sensitive to visual imaging, and can be applied For pinhole and fisheye cameras, it is not sensitive to modules with different parameters (such as camera external parameters), and has good mass production characteristics; and the physical modeling scheme is used, which has strong interpretability.

The following describes the method for determining the orientation of a target object by taking the application of the method for determining the orientation of a target object provided in this specification in the automatic driving of a vehicle as an example, which specifically includes the following steps.

Step 502 : Receive a video frame including a vehicle, and obtain a full-vehicle detection frame, key points, and vehicle type of the vehicle in the video frame.

Specifically, the full-vehicle detection frame, key points, and model of the vehicle in the video frame are obtained respectively according to the three deep learning models.

Step 504: Obtain the initial length and width of the vehicle in the video frame according to the vehicle type.

Step 506: Under the constraint of the initial length and width of the vehicle, determine the real length and width of the vehicle according to the key points of the vehicle and the acquired camera extrinsic parameters of the video frame of the vehicle.

Step 508: Calculate the initial orientation of the vehicle in the video frame according to the length and width of the vehicle and the mapping key points of the entire vehicle detection frame of the vehicle in the world coordinate system.

Step 510: Establish a motion model of the bicycle for the vehicle in advance, obtain the predicted orientation of the vehicle in the video frame, and use the Kalman filter to estimate the initial orientation and the predicted orientation.

Step 512: After weighted fusion of the initial orientation and the predicted orientation using the Kalman filter, the stable orientation of the vehicle is obtained.

The method for determining the orientation of the target object provided in the embodiments of this specification is applied to automatic driving of vehicles. By modeling the constraint relationship between the length, width and orientation of the vehicle, a stable and accurate orientation of the vehicle can be obtained, and for stationary vehicles It can be applied to running vehicles, and it is also applicable to vehicles near and far. There is no limitation in principle, and the overall performance is better. Then, because the model used is a key point model, it has a strong image texture, which is better than the original one. The large model has advantages in model stability and long tail problems. Secondly, by disassembling the orientation calculation into multiple parts such as model estimation, length and width calculation, Kalman filtering, etc., it can be divided and conquered for decoupling and fusion, and finally achieve a robust In the end, the overall orientation calculation model of the vehicle only depends on the key points, and the amount of calculation itself is relatively small. Therefore, the method for determining the orientation of the target object provided by the embodiments of this specification can be used in accuracy, stability, application range, real-time performance. All have better effects.

Corresponding to the above method embodiments, the present specification also provides an embodiment of an apparatus for determining the orientation of a target object. FIG. 6 shows a schematic structural diagram of an apparatus for determining the orientation of a target object provided by an embodiment of the present specification. As shown in Figure 6, the device includes:

The first video receiving module 602 is configured to receive the i+1 th video frame including the target object, and obtain the shape, key points and edge detection frame of the target object in the i+1 th video frame;

a first determining module 604, configured to determine a target attribute value of the target object based on the shape and key points of the target object;

The second determination module 606 is configured to determine the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame;

The first target orientation determination module 608 is configured to determine a second orientation of the target object in the i+1 th video frame based on the target orientation of the target object in the ith video frame, based on the first orientation and the second orientation. The orientation determines the orientation of the target object in the i+1 th video frame.

Optionally, the device further includes:

The second video receiving module is configured to receive the ith video frame containing the target object, and obtain the shape, key points and edge detection frame of the target object in the ith video frame;

a third determining module, configured to determine a target attribute value of the target object based on the shape and key points of the target object;

The second target orientation determination module is configured to determine the target orientation of the target object in the ith video frame according to the target attribute value of the target object and the edge detection frame.

Optionally, the first video receiving module 602 is further configured as:

The i+1 th video frame is input into the first recognition model, the second recognition model and the third recognition model respectively, and the shape, key points and edge detection frame of the target object in the i+1 th video frame are acquired.

Optionally, the third determining module is further configured to:

Determine the first initial attribute value of the target object in the ith video frame based on the shape of the target object in the ith video frame;

Determine the second initial attribute value of the target object in the i-th video frame based on the key points of the target object in the i-th video frame and the camera extrinsic parameters for obtaining the i-th video frame;

In the case that the second initial attribute value is less than or equal to the first initial attribute value, the second initial attribute value is used as the target attribute value of the target object in the ith video frame.

Optionally, the first determining module 604 is further configured to:

Based on the shape of the target object in the i+1 th video frame, determine the first initial attribute value of the target object in the i+1 th video frame;

Determine the second initial attribute of the target object in the i+1 th video frame based on the key points of the target object in the i+1 th video frame and the camera extrinsic parameters used to obtain the i+1 th video frame value;

In the case that the second initial attribute value is less than or equal to the first initial attribute value, determine the target object in the i+1 th video frame based on the target attribute value of the target object in the i th video frame The third initial attribute value of ;

The target attribute value of the target object in the i+1 th video frame is determined based on the second initial attribute value and the third initial attribute value.

Optionally, the second determining module 606 is further configured to:

Mapping the edge detection frame of the target object in the i+1 th video frame in a preset coordinate system to obtain the mapping key point of the edge detection frame of the target object in the i+1 th video frame;

The first orientation of the target object in the i+1 th video frame is determined based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object.

Optionally, the second determining module 606 is further configured to:

Determine the first target edge and the second target edge of the target object based on the edge detection frame;

Determine the edge value of the first target edge and the edge value of the second target edge according to the target attribute value of the target object;

Based on the coordinate value of the mapping key point in the preset coordinate system, the edge value of the first target edge and the edge value of the second target edge, calculate and obtain the i+1 th video frame. The first orientation of the target object.

Optionally, the second target orientation determination module is further configured as:

The edge detection frame of the target object in the i-th video frame is mapped in a preset coordinate system, and the mapping key point of the edge detection frame of the target object in the i-th video frame is obtained;

Determine the first orientation of the target object in the i-th video frame based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object;

In the case where i is 1, the first orientation of the target object in the i-th video frame is used as the target orientation of the target object in the i-th video frame.

Optionally, the second target orientation determination module is further configured as:

Determine the first target edge and the second target edge of the target object based on the edge detection frame;

Determine the edge value of the first target edge and the edge value of the second target edge according to the target attribute value of the target object;

Based on the coordinate value of the mapping key point in the preset coordinate system, the edge value of the first target edge and the edge value of the second target edge, calculate and obtain the target object in the i-th video frame the first orientation.

Optionally, the first target orientation determination module 608 is further configured to:

Input the target orientation of the target object in the i-th video frame into the bicycle model, and obtain the second orientation of the target object in the i+1-th video frame.

An embodiment of this specification implements a device for determining the orientation of a target object, which is realized by decomposing the calculation of the target orientation into multiple steps such as shape estimation, target attribute value calculation, and orientation calculation of the target object, so that each implementation step can be well adjusted. The decoupling and fusion of the target object can finally obtain the accurate and stable target orientation of the target object.

The above is a schematic solution of an apparatus for determining the orientation of a target object according to this embodiment. It should be noted that the technical solution of the device for determining the orientation of the target object and the technical solution of the above-mentioned method for determining the orientation of the target object belong to the same concept. A description of the technical solution towards the determination method.

The device for determining the orientation of a target object provided in the embodiments of this specification can be applied to an automatic driving scenario, and the functional modules (eg, the first determination module, the second determination module, and the first target orientation determination module, etc.) The orientation of the autonomous vehicle is calculated.

Of course, these algorithmic modules (such as the above-mentioned functional modules) will vary depending on the type of autonomous vehicle. For example, different algorithm modules will be involved for logistics vehicles, public service vehicles, medical service vehicles, and terminal service vehicles. The following are examples of the algorithm modules for these four types of autonomous vehicles:

Among them, the logistics vehicle refers to the vehicle used in the logistics scenario, for example, it may be a logistics vehicle with an automatic sorting function, a logistics vehicle with a refrigeration and heat preservation function, and a logistics vehicle with a measurement function. These logistics vehicles will involve different algorithm modules.

For example, a logistics vehicle can be equipped with an automated sorting device, which can automatically take out and transport, sort, and store the goods after the logistics vehicle arrives at the destination. This involves an algorithm module for cargo sorting, which mainly implements logical control of cargo take-out, handling, sorting, and storage.

For another example, for the cold chain logistics scenario, the logistics vehicle can also be equipped with a refrigeration and heat preservation device, which can realize the refrigeration or heat preservation of the transported fruits, vegetables, aquatic products, frozen food and other perishable food, so that they are in Appropriate temperature environment to solve the problem of long-distance transportation of perishable food. This involves an algorithm module for refrigeration and heat preservation control. The algorithm module is mainly used to dynamically and adaptively calculate the appropriateness of cold meals or heat preservation according to information such as food (or item) properties, perishability, transportation time, current season, and climate. According to the appropriate temperature, the refrigeration and heat preservation device is automatically adjusted, so that the transport personnel do not need to manually adjust the temperature when the vehicle transports different foods or items, which frees the transport personnel from the tedious temperature control and improves the efficiency of refrigeration and heat preservation transportation.

For another example, in most logistics scenarios, charges are based on the volume and/or weight of the package, and the number of logistics packages is very large, and it is very inefficient to rely solely on the courier to measure the volume and/or weight of the package, resulting in labor costs. higher. Therefore, in some logistics vehicles, a measuring device is added, which can automatically measure the volume and/or weight of the logistics package and calculate the cost of the logistics package. This involves an algorithm module for logistics package measurement, which is mainly used to identify the type of logistics package, determine the measurement method of the logistics package, such as volume measurement or weight measurement, or a combination of volume and weight measurement at the same time, and The measurement of volume and/or weight can be done according to the determined measurement method, and the cost calculation can be done according to the measurement result.

Among them, the public service vehicle refers to a vehicle that provides some kind of public service, such as a fire truck, a deicer, a sprinkler, a snow plow, a garbage disposal vehicle, a traffic command vehicle, and the like. These public service vehicles will involve different algorithm modules.

For example, for an autonomous fire truck, its main task is to carry out a reasonable fire-fighting task at the fire scene, which involves an algorithm module for fire-fighting tasks. Logic such as automatic control of fire extinguishing device.

For another example, for a de-icing vehicle, its main task is to remove the ice and snow on the road, which involves an algorithm module for de-icing, which at least needs to identify the ice and snow conditions on the road, and formulate a de-icing plan according to the ice and snow conditions, such as Which road sections need to be de-iced, which sections do not need to be de-iced, whether to use the salt spray method, the number of grams of salt, and the logic of automatic control of the de-icing device when the de-icing plan is determined.

Among them, medical service vehicles refer to self-driving vehicles that can provide one or more medical services. Such vehicles can provide medical services such as disinfection, temperature measurement, dispensing, and isolation. This involves algorithm modules that provide various self-service medical services. , these algorithm modules mainly realize the identification of disinfection needs and the control of the disinfection device, so that the disinfection device can disinfect the patient, or identify the patient's position, and control the temperature measurement device to automatically measure the temperature of the patient close to the patient's forehead, or , used to realize the judgment of the disease, give the prescription according to the judgment result and need to realize the identification of the medicine/drug container, and control the medicine taking manipulator, so that it can grab the medicine for the patient according to the prescription, and so on.

Among them, terminal service vehicles refer to self-service autonomous vehicles that can replace some terminal devices to provide users with certain convenient services. For example, these vehicles can provide users with services such as printing, attendance, scanning, unlocking, payment, and retail.

For example, in some application scenarios, users often need to go to a specific location to print or scan documents, which is time-consuming and labor-intensive. As a result, there is a terminal service vehicle that can provide users with printing/scanning services. These service vehicles can be interconnected with user terminal equipment. The user sends a print command through the terminal device, and the service vehicle responds to the print command, automatically prints the documents required by the user and automatically prints the documents required by the user. The printed documents can be automatically sent to the user's location, and the user does not need to queue at the printer, which can greatly improve the printing efficiency. Alternatively, the user can move to the user's location in response to the scanning instruction issued by the user through the terminal device, and the user can place the document to be scanned on the scanning tool of the service vehicle to complete the scanning, without queuing at the printing/scanning machine, saving time and effort. This involves an algorithm module for providing printing/scanning services. The algorithm module needs at least to identify the interconnection with the user terminal equipment, the response to the printing/scanning command, the positioning of the user's position, and the travel control.

For another example, with the development of new retail business, more and more e-commerce merchants use self-service vending machines to sell goods to major office buildings and public areas, but these self-service vending machines are placed in fixed positions and cannot be moved. The user needs to go to the self-service vending machine to buy the desired product, and the convenience is still poor. As a result, there are self-driving vehicles that can provide retail services. These service vehicles can carry goods and move automatically, and can provide corresponding self-service shopping APPs or shopping portals. Users can provide retail services through APPs or shopping portals with the help of terminals such as mobile phones. After the vehicle receives the order request, it can determine whether the current remaining goods have the goods purchased by the user and whether the quantity is sufficient. After determining When there are goods purchased by the user and the quantity is sufficient, these goods can be automatically moved to the user's location, and these goods are provided to the user, which further improves the convenience of the user's shopping, saves the user's time, and allows the user to use the time for more important things. This involves algorithm modules that provide retail services. These algorithm modules mainly implement logic such as responding to user order requests, order processing, product information maintenance, user location positioning, and payment management.

Referring to FIG. 7, FIG. 7 shows a structural block diagram of a computing device 700 provided according to an embodiment of the present specification. Components of the computing device 700 include, but are not limited to, memory 710 and processor 720 . The processor 720 is connected with the memory 710 through the bus 730, and the database 750 is used for storing data.

Computing device 700 also includes access device 740 that enables computing device 700 to communicate via one or more networks 760 . Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. Access device 740 may include one or more of any type of network interface (eg, a network interface card (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, World Interoperability for Microwave Access ( Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, and the like.

In one embodiment of the present specification, the above-described components of computing device 700 and other components not shown in FIG. 7 may also be connected to each other, such as through a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 7 is only for the purpose of example, rather than limiting the scope of the present specification. Those skilled in the art can add or replace other components as required.

Computing device 700 may be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (eg, tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (eg, smart phones) ), wearable computing devices (eg, smart watches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. Computing device 700 may also be a mobile or stationary server.

The processor 720 is configured to execute the following computer-executable instructions, the processor is configured to execute the computer-executable instructions, and when the computer-executable instructions are executed by the processor, implement the steps of the method for determining the orientation of the target object.

The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned target object orientation determination method belong to the same concept, and the details that are not described in detail in the technical solution of the computing device can be referred to the above-mentioned technical solution of the target object orientation determination method. description of.

An embodiment of the present specification further provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, implements the steps of the method for determining the orientation of the target object.

The above is a schematic solution of a computer-readable storage medium of this embodiment. It should be noted that the technical solution of the storage medium and the above-mentioned technical solution of the target object orientation determination method belong to the same concept, and the details that are not described in detail in the technical solution of the storage medium can be referred to the above-mentioned technical solution of the target object orientation determination method. description of.

The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The computer instructions include computer program code, which may be in source code form, object code form, an executable file, some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Electric carrier signals and telecommunication signals are not included.

It should be noted that, for the convenience of description, the foregoing method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of this specification are not limited by the described action sequences. Limitation, because certain steps may be performed in other orders or simultaneously according to embodiments of the present specification. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily all necessary for the embodiments of the specification.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are provided only to aid in the elaboration of the present specification. Alternative embodiments are not intended to exhaust all details, nor do they limit the invention to only the described embodiments. Obviously, many modifications and changes can be made in accordance with the contents of the embodiments of the present specification. These embodiments are selected and described in this specification to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims (12)

1. A method for determining the orientation of a target object, comprising: Receive the i+1 th video frame including the target object, and obtain the shape, key points and edge detection frame of the target object in the i+1 th video frame; Determine the target attribute value of the target object based on the shape and key points of the target object; Determine the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame; The second orientation of the target object in the i+1 th video frame is determined based on the target orientation of the target object in the ith video frame, and the i+1 th video frame is determined based on the first orientation and the second orientation The target orientation of the target object in . 2. The method for determining the orientation of a target object according to claim 1, before receiving the i+1 th video frame including the target object, further comprising: Receive the ith video frame containing the target object, and obtain the shape, key points and edge detection frame of the target object in the ith video frame; Determine the target attribute value of the target object based on the shape and key points of the target object; The target orientation of the target object in the ith video frame is determined according to the target attribute value of the target object and the edge detection frame. 3. The method for determining the orientation of a target object according to claim 1 or 2, wherein the acquisition of the shape, key points and edge detection frame of the target object in the i+1 th video frame comprises: The i+1 th video frame is input into the first recognition model, the second recognition model and the third recognition model respectively, and the shape, key points and edge detection frame of the target object in the i+1 th video frame are acquired. 4. The method for determining the orientation of a target object according to claim 2, wherein determining the target attribute value of the target object based on the shape and key points of the target object, comprising: Determine the first initial attribute value of the target object in the ith video frame based on the shape of the target object in the ith video frame; Determine the second initial attribute value of the target object in the i-th video frame based on the key points of the target object in the i-th video frame and the camera extrinsic parameters for obtaining the i-th video frame; In the case that the second initial attribute value is less than or equal to the first initial attribute value, the second initial attribute value is used as the target attribute value of the target object in the ith video frame. 5. The method for determining the orientation of a target object according to claim 4, wherein determining the target attribute value of the target object based on the shape and key points of the target object, comprising: Based on the shape of the target object in the i+1 th video frame, determine the first initial attribute value of the target object in the i+1 th video frame; Determine the second initial attribute of the target object in the i+1 th video frame based on the key points of the target object in the i+1 th video frame and the camera extrinsic parameters used to obtain the i+1 th video frame value; In the case that the second initial attribute value is less than or equal to the first initial attribute value, determine the target object in the i+1 th video frame based on the target attribute value of the target object in the i th video frame The third initial attribute value of ; The target attribute value of the target object in the i+1 th video frame is determined based on the second initial attribute value and the third initial attribute value. 6. The method for determining the orientation of a target object according to claim 1, wherein determining the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and an edge detection frame, comprising: Mapping the edge detection frame of the target object in the i+1 th video frame in a preset coordinate system to obtain the mapping key point of the edge detection frame of the target object in the i+1 th video frame; The first orientation of the target object in the i+1 th video frame is determined based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object. 7 . The method for determining the orientation of a target object according to claim 6 , wherein the i+1th is determined based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object. 8 . The first orientation of the target object in video frames, including: Determine the first target edge and the second target edge of the target object based on the edge detection frame; Determine the edge value of the first target edge and the edge value of the second target edge according to the target attribute value of the target object; Based on the coordinate value of the mapping key point in the preset coordinate system, the edge value of the first target edge and the edge value of the second target edge, calculate and obtain the i+1 th video frame. The first orientation of the target object. 8. The method for determining the direction of a target object according to claim 2, wherein the target direction of the target object in the i-th video frame is determined according to the target attribute value of the target object and an edge detection frame, comprising: The edge detection frame of the target object in the i-th video frame is mapped in a preset coordinate system, and the mapping key point of the edge detection frame of the target object in the i-th video frame is obtained; Determine the first orientation of the target object in the i-th video frame based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object; In the case where i is 1, the first orientation of the target object in the i-th video frame is used as the target orientation of the target object in the i-th video frame. 9. The method for determining the orientation of a target object according to claim 8, wherein the i-th video is determined based on the coordinate value of the mapping key point in the preset coordinate system and the target attribute value of the target object The first orientation of the target object in the frame, including: Determine the first target edge and the second target edge of the target object based on the edge detection frame; Determine the edge value of the first target edge and the edge value of the second target edge according to the target attribute value of the target object; Based on the coordinate value of the mapping key point in the preset coordinate system, the edge value of the first target edge and the edge value of the second target edge, calculate and obtain the target object in the i-th video frame the first orientation. 10. A device for determining the orientation of a target object, comprising: The first video receiving module is configured to receive the i+1 th video frame including the target object, and obtain the shape, key points and edge detection frame of the target object in the i+1 th video frame; a first determining module, configured to determine a target attribute value of the target object based on the shape and key points of the target object; a second determining module, configured to determine the first orientation of the target object in the i+1 th video frame according to the target attribute value of the target object and the edge detection frame; a first target orientation determination module configured to determine a second orientation of the target object in the i+1 th video frame based on the target orientation of the target object in the i th video frame, based on the first orientation and the second orientation Determine the target orientation of the target object in the i+1 th video frame. 11. A computing device comprising: memory and processor; The memory is used for storing computer-executable instructions, the processor is used for executing the computer-executable instructions, and when the computer-executable instructions are executed by the processor, the method for determining the orientation of a target object according to any one of claims 1-9 is implemented A step of. 12. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for determining the orientation of a target object according to any one of claims 1-9.

Images