WO2023016271A1 - 位姿确定方法、电子设备及可读存储介质 - Google Patents

位姿确定方法、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2023016271A1
WO2023016271A1 PCT/CN2022/108678 CN2022108678W WO2023016271A1 WO 2023016271 A1 WO2023016271 A1 WO 2023016271A1 CN 2022108678 W CN2022108678 W CN 2022108678W WO 2023016271 A1 WO2023016271 A1 WO 2023016271A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target vehicle
historical
yaw angle
target
Prior art date
Application number
PCT/CN2022/108678
Other languages
English (en)
French (fr)
Inventor
王彬
Original Assignee
北京迈格威科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京迈格威科技有限公司 filed Critical 北京迈格威科技有限公司
Publication of WO2023016271A1 publication Critical patent/WO2023016271A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/06—Topological mapping of higher dimensional structures onto lower dimensional surfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Definitions

  • the present application relates to the field of image processing, and in particular, relates to a pose determination method, device, electronic equipment, and readable storage medium.
  • the determined vehicle pose can be used in scenarios such as counting traffic flow and judging whether a driver is driving illegally.
  • the 3D model of the vehicle can be used to restore the 3D pose information of the vehicle from the collected 2D images.
  • the commonly used method is to optimize the reprojection error between the 3D projection point of the vehicle and the key points of the vehicle in the 2D image through the least squares algorithm, and the corresponding pose information can be calculated according to the minimum reprojection error value Determine the current pose of the vehicle. In this way, the calculation process takes a lot of time.
  • the embodiment of the present application provides a pose determination method, electronic equipment, and a readable storage medium, which are used to determine the current actual pose of the target vehicle based on the determined estimated pose, and can quickly determine the corresponding pose of the target vehicle at the time of shooting. actual pose.
  • Some embodiments of the present application provide a pose determination method, which may include: acquiring an image to be processed; the image to be processed includes an image of a target vehicle; based on historical yaw angle information of the target vehicle, determining the The estimated pose of the target vehicle at the moment when the image to be processed is taken; the historical yaw angle information is the yaw angle information of the target vehicle at the time before the moment when the image to be processed is taken; verify the estimated position Whether the pose satisfies the verification condition; and if so, determining the estimated pose as the actual pose of the target vehicle at the moment when the image to be processed is captured.
  • the calculation process for determining the actual pose can be shortened, and the number of multiplexing channels of the system can be increased.
  • the estimated pose may include estimated yaw angle information and estimated position information; and based on the historical yaw angle information of the target vehicle, determining the position of the target vehicle at the time when the image to be processed is captured
  • Estimating the pose may include: determining the estimated yaw angle information of the target vehicle based on the historical yaw angle information of the target vehicle at a previous moment; and/or based on the target vehicle in the image to be processed.
  • the detection results of the two-dimensional key points of the vehicle determine the estimated position information of the target vehicle. In this way, based on the fact that the moving direction of the target vehicle does not change in a short period of time, the estimated yaw angle information can be determined based on the historical yaw angle information at the last moment, and the estimation process is more reasonable.
  • the determining the estimated yaw angle information of the target vehicle based on the historical yaw angle information of the target vehicle at the last moment may include: determining the historical yaw angle information of the last moment is the estimated yaw angle information; or determine the yaw angle obtained after compensating the target angle on the basis of the historical yaw angle at the last moment as the estimated yaw angle information.
  • the target angle may be determined based on the frequency of images captured by the camera and the speed of the target vehicle.
  • the determining the estimated position information of the target vehicle based on the detection result of the two-dimensional key points of the target vehicle in the image to be processed may include: if the target vehicle is detected in the image to be processed To the target two-dimensional key point of the target vehicle, determine the image coordinates of the target two-dimensional key point; according to the image coordinates of the target two-dimensional key point and the vehicle three-dimensional model matched with the target vehicle, use the projection formula Determine the target world coordinate information of the target two-dimensional key point in the world coordinate system, and use the target world coordinate information as the estimated position information; the world coordinate system includes the target vehicle's motion plane as the coordinate The coordinate system of the surface.
  • the estimated position information can be determined directly based on the detected image coordinates of the two-dimensional key points of the target, and the estimation process is simple and intuitive.
  • the target two-dimensional key points may include key points corresponding to a front logo or a left-view mirror on the target vehicle.
  • the determining the estimated position information of the target vehicle based on the detection result of the two-dimensional key points of the target vehicle in the image to be processed may include: if there is no The target two-dimensional key point of the target vehicle is detected, and the estimated position information is determined based on the estimated yaw angle information and the vehicle three-dimensional model matched with the target vehicle. In this way, when the target two-dimensional key point is not detected, the estimated position information can also be determined.
  • the determining the estimated position information based on the estimated yaw angle information and the vehicle three-dimensional model matched with the target vehicle may include: determining image coordinates of candidate two-dimensional key points, the alternative Selecting two-dimensional key points is at least one of the two-dimensional key points that can be detected for two-dimensional key point detection on the image to be processed; according to the image coordinates of the candidate two-dimensional key points and the three-dimensional model of the vehicle , using a projection formula to determine the candidate world coordinate information of the candidate two-dimensional key point in the world coordinate system; the world coordinate system includes a coordinate system with the motion plane of the target vehicle as the coordinate plane; according to the target model point The relative position relationship with the candidate model point, the estimated yaw angle information and the candidate world coordinate information determine the estimated position information; the target model point is the target two-dimensional key point at the The corresponding model point in the vehicle three-dimensional model, the candidate model point is the corresponding model point of the candidate two-dimensional key point in the vehicle three-dimensional model.
  • estimated position information based
  • the candidate two-dimensional key points may be determined in the following manner: using a convolutional neural network to perform two-dimensional key point detection on the image to be processed to obtain the image coordinates and confidence of each two-dimensional key point; The two-dimensional key point with the highest confidence is taken as the candidate two-dimensional key point.
  • the verifying whether the estimated pose satisfies the verification condition may include: determining the projected image coordinates of a matching model point under the estimated pose, the matching model point being a vehicle that matches the target vehicle A model point corresponding to the two-dimensional key point of the target vehicle detected in the image to be processed in the three-dimensional model; based on the projected image coordinates of the matching model point and the two-dimensional key point detected in the image to be processed The image coordinates of the points are used to calculate a reprojection error value; and it is judged whether the reprojection error value is smaller than a first error threshold. In this way, the estimated pose that satisfies the verification conditions is more consistent with the actual pose.
  • the method may further include: if the estimated pose does not meet the verification condition, based on the two-dimensional key point information, the target The three-dimensional model of the vehicle matched with the vehicle, using a projection formula to determine the actual pose; the two-dimensional key point information is obtained by performing two-dimensional key point detection on the target vehicle in the image to be processed.
  • the estimated pose does not meet the verification conditions, other methods can be used to determine the actual pose, and this method is combined with the estimation method, so that the actual pose can be obtained regardless of whether the estimated pose meets the verification conditions.
  • the historical yaw angle information may be determined according to historical images, the historical yaw angle information is in one-to-one correspondence with the historical images, and the shooting time of the historical images is earlier than that of the image to be processed.
  • the historical image includes the image of the target vehicle, and the historical image and the image to be processed are taken by the same camera; the first historical yaw angle information corresponding to the earliest historical image at the shooting moment can be based on the following steps Determining: performing two-dimensional key point detection on the target vehicle in the earliest historical image at the shooting time to obtain the image coordinates of the historical two-dimensional key point; determining the historical matching model point according to the historical two-dimensional key point, and the historical matching model The point is a model point corresponding to the historical two-dimensional key point in the vehicle three-dimensional model matched with the target vehicle; determine the initial estimated pose, and use the initial estimated pose as the current estimated pose; use the least squares method Update the current estimated pose and optimize the reprojection error value between the projected image coordinate
  • Some other embodiments of the present application also provide an electronic device, which may include a processor and a memory, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor , execute the steps in the method as provided in some embodiments of the above-mentioned application.
  • Some other embodiments of the present application also provide a computer program product, which may include a computer program, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the above-mentioned Steps in the methods provided in some embodiments of the present application.
  • FIG. 1 is a flow chart of a pose determination method provided in an embodiment of the present application
  • FIG. 2 is a flow chart of another pose determination method provided in the embodiment of the present application.
  • FIG. 3 is a schematic diagram of an application scenario for determining an estimated pose involved in the present application
  • FIG. 4 is a structural block diagram of a pose determining device provided in an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of an electronic device for performing a pose determination method provided by an embodiment of the present application.
  • Artificial Intelligence is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence.
  • the subject of artificial intelligence is a comprehensive subject that involves many technologies such as chips, big data, cloud computing, Internet of Things, distributed storage, deep learning, machine learning, and neural networks.
  • computer vision is specifically to allow machines to recognize the world.
  • Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian detection, etc.
  • the present application provides a pose determination method, device, electronic equipment and readable storage medium. Utilizing the characteristic that the vehicle’s motion direction does not change much in a short period of time, the historical yaw angle information of the target vehicle is determined as the estimated pose of the target vehicle at the shooting moment, and the estimated pose is verified through the pre-set verification conditions.
  • the technical solution can quickly determine the actual pose of the target vehicle at the time of shooting, increasing the number of multiplexing channels of the system.
  • the above pose determination method can be applied to data processing centers such as servers and cloud platforms that can essentially provide computing, information transmission and storage functions.
  • the present application uses the server as an example to introduce the above pose determination method in detail.
  • FIG. 1 shows a flow chart of a method for determining a pose provided by an embodiment of the present application.
  • the pose determination method may include the following steps 101 to 103 .
  • Step 101 the server acquires an image to be processed;
  • the image to be processed includes an image of a target vehicle;
  • the above-mentioned target vehicles may include, for example, trucks, vans, cars and other types of vehicles.
  • the server may obtain the above image to be processed.
  • the image to be processed may be, for example, an image intercepted from a video including the target vehicle, or an image including the target vehicle captured by a camera.
  • Step 102 based on the historical yaw angle information of the target vehicle, the server determines the estimated pose of the target vehicle at the time when the image to be processed is captured; the historical yaw angle information is the Processing the yaw angle information at the time before the image capture time;
  • pose may include position and pose.
  • attitude can be expressed in terms of yaw angle.
  • the historical yaw angle information of the target vehicle can be obtained.
  • the historical yaw angle information is the yaw angle information of the target vehicle determined according to the images captured before the image to be processed.
  • the yaw angle information may be in one-to-one correspondence with the shooting time of the image, for example, a yaw angle corresponding to a unique shooting time may be determined for an image.
  • the historical yaw angle information may include one or more yaw angle information determined in one or more images taken before the image to be processed.
  • the reprojection error between the projection point of the 3D model of the vehicle and the 2D key points in the image to be processed can be optimized by the least squares method, and the first yaw angle of the target vehicle can be determined.
  • the first yaw angle information of the target vehicle is obtained.
  • the estimated yaw angle information may be used as the second yaw angle information and historical yaw angle information corresponding to the later determined yaw angle information.
  • the second yaw angle information and the subsequently determined yaw angle information respectively correspond to the second captured image and the subsequent captured images.
  • the preset initial value of the yaw angle may also be determined as the historical yaw angle information.
  • the initial value of the yaw angle can be set according to the actual situation, such as 5°, 30°, etc.
  • the estimated pose of the target vehicle at the moment when the image to be processed can be determined can be determined.
  • the above-mentioned estimated pose may include, for example, position information and yaw angle information of the target vehicle.
  • the above position information may be characterized by, for example, the coordinate information of the target vehicle in the world coordinate system.
  • the above-mentioned estimated pose can include, for example, the coordinate information and yaw angle information of the target vehicle in the world coordinate system (generally the earth coordinate system established with the road surface as the coordinate plane), which can be coordinates (X, Y, ⁇ ) characterization.
  • X, Y can represent any coordinate value within the scope of the world coordinate system
  • ⁇ can represent any degree within the range of (0,2 ⁇ ).
  • Step 103 the server verifies whether the estimated pose satisfies the verification condition; and if so, determines the estimated pose as the actual pose of the target vehicle at the time when the image to be processed is captured.
  • the estimated pose of the target vehicle After the estimated pose of the target vehicle is determined, it can be verified whether the estimated pose satisfies the verification condition.
  • the estimated pose may be determined as the actual pose corresponding to the current target vehicle. For example, after verifying that the estimated pose (4, 5, 30°) of the target vehicle satisfies the verification condition, (4, 5, 30°) can be determined as the current actual pose of the target vehicle.
  • the actual pose of the target vehicle can be estimated based on the historical yaw angle information of the target vehicle, and then the actual pose of the target vehicle can be shortened.
  • the calculation process of the pose increases the number of multiplexing channels of the system.
  • FIG. 2 shows a flowchart of another method for determining a pose provided by an embodiment of the present application.
  • the pose determination method may include the following steps 201 to 204 .
  • Step 201 the server acquires the image to be processed; the image to be processed includes the image of the target vehicle;
  • step 201 may be the same as or similar to the above step 101, and will not be repeated here.
  • Step 202 the server determines the estimated yaw angle information of the target vehicle based on the historical yaw angle information of the target vehicle at a previous moment;
  • the vehicle when the vehicle is driving on the road, except for a few that need to change lanes, it will drive in the current lane, and the yaw angle will not change in a short period of time (such as 1 second, 3 seconds, 20 seconds, etc.). big change. For example, when the vehicle is driving at a constant speed or accelerating on a straight lane, its yaw angle will not change greatly within a certain period of time; when the vehicle is driving on a curved road, when the interval is short enough, Its yaw angle will not change greatly.
  • a short period of time such as 1 second, 3 seconds, 20 seconds, etc.
  • the estimated yaw angle information of the target vehicle at the current moment can be estimated based on the historical yaw angle information.
  • the above step 202 may include: determining the historical yaw angle information at the last moment as the estimated yaw angle information.
  • the target vehicle does not change lanes, it can be roughly considered that the target vehicle is driving straight in a short period of time, and then its historical yaw angle at the previous moment can be determined as the estimated yaw angle at the current moment to simplify the estimation Estimation process of yaw angle information.
  • the above step 202 may include: determining a yaw angle obtained after compensating a certain angle based on the historical yaw angle at the last moment as the estimated yaw angle information.
  • the compensation angle can be estimated from the frequency of images captured by the camera and the speed of the target vehicle, and then the compensation angle can be added to the historical yaw at the previous moment In order to determine the yaw angle information obtained at this time as the estimated yaw angle information.
  • the above “xx” may be within a reasonable range such as "25°”, “15°” and the like.
  • Step 203 the server determines the estimated position information of the target vehicle based on the detection result of the two-dimensional key points of the target vehicle in the image to be processed.
  • the two-dimensional key points may include, for example, key points corresponding to the front logo on the target vehicle, the left-view mirror of the target vehicle, and the like.
  • convolutional neural networks for example, convolutional neural networks, heat maps, etc. can be used to detect two-dimensional key points.
  • the above step 203 may include step 2031: if the target two-dimensional key point of the target vehicle is detected in the image to be processed, determine the image coordinates of the target two-dimensional key point; The image coordinates of the target two-dimensional key points and the vehicle three-dimensional model matched with the target vehicle, using the projection formula to determine the target world coordinate information of the target two-dimensional key points in the world coordinate system, and the target world Coordinate information is used as the estimated position information; the world coordinate system includes a coordinate system with the motion plane of the target vehicle as a coordinate plane.
  • the image coordinates of the target two-dimensional key point in the image to be processed can be determined by using a convolutional neural network such as the above, and then can be Based on the image coordinates, the target world coordinate information of the target two-dimensional key points in the world coordinate system is determined by using the projection formula and the three-dimensional model of the vehicle.
  • the three-dimensional vehicle model here can be obtained, for example, by matching in a preset model library after identifying the target vehicle in the image to be processed.
  • the aforementioned preset model library may include, for example, three-dimensional models of vehicles corresponding to vans, cars, and trucks, respectively.
  • the vehicle three-dimensional model corresponding to the target vehicle can be obtained by identifying the target vehicle in the image to be processed.
  • information such as the length, width, height, and relative positional relationship between each model point of the target vehicle can be determined through the 3D vehicle model. Therefore, the position of each model point in the 3D vehicle model can be represented by, for example, the model point corresponding to the target 2D key point as a reference point, or can be represented by using other model points as a reference point.
  • the height information of the target vehicle can be determined based on the known three-dimensional model of the vehicle, and then the target vehicle can be calculated by using the projection formula
  • the target world coordinate information in the world coordinate system is (x w , y w , z w ).
  • the above u, v, x w , y w , z w can represent any number in their own coordinate system, where z w is known.
  • the projection formula above may include, for example:
  • s is the scale factor
  • s is the internal parameter matrix of the camera
  • the world coordinate system takes the target vehicle’s motion plane (such as the road surface) as the coordinate plane, and when the vehicle 3D model of the target vehicle is known, the target 2D key points in the world coordinate system can be The height information is considered known (ie, z w is known). In this way, when using the above projection formula for calculation, a binary quadratic equation with a unique solution can be obtained. Then the only target world coordinate information can be obtained.
  • the above step 203 may include step 2032: if no target two-dimensional key point of the target vehicle is detected in the image to be processed, based on the estimated yaw angle information and the The target vehicle is matched with a three-dimensional vehicle model to determine the estimated position information.
  • the estimated position information may be determined based on the estimated yaw angle information and the three-dimensional model of the vehicle. For example, when the determined target two-dimensional key point is the left-view mirror of the target vehicle, if the image to be processed is obtained based on the camera being located in the right front of the target vehicle, if the left-view mirror cannot be detected in the image to be processed, Then the current detection result is that the target two-dimensional key point is not detected in the image to be processed. The estimated position information can then be estimated by estimating the yaw angle and the 3D model of the vehicle.
  • determining the estimated position information based on the estimated yaw angle information and the vehicle three-dimensional model matched with the target vehicle in the step 2032 may include the following substeps:
  • Sub-step 1 determining the image coordinates of a candidate two-dimensional key point, the candidate two-dimensional key point is at least one of the two-dimensional key points that can be detected by performing two-dimensional key point detection on the image to be processed;
  • multiple two-dimensional key points may be detected, and at this time, one of the detected two-dimensional key points may be determined as a candidate two-dimensional key point. Then, the image coordinates of the candidate 2D key can be determined.
  • a convolutional neural network can be used to perform two-dimensional key point detection on the image to be processed, so that the image coordinates and confidence of each two-dimensional key point can be obtained.
  • the two-dimensional key point with the highest confidence can be selected as a candidate two-dimensional key point, which can also improve the confidence of the estimated position information to a certain extent.
  • Sub-step 2 according to the image coordinates of the candidate two-dimensional key points and the three-dimensional model of the vehicle, use a projection formula to determine the candidate world coordinate information of the candidate two-dimensional key points in the world coordinate system;
  • the world The coordinate system includes a coordinate system with the motion plane of the target vehicle as a coordinate plane;
  • the implementation process of the above-mentioned sub-step 2 may be similar to the implementation process of the above-mentioned step 2031, which will not be repeated here.
  • Sub-step 3 determine the estimated position information according to the relative positional relationship between the target model point and the candidate model point, the estimated yaw angle information and the candidate world coordinate information; the target model point is the The target two-dimensional key point is a model point corresponding to the vehicle three-dimensional model, and the candidate model point is a model point corresponding to the candidate two-dimensional key point in the vehicle three-dimensional model.
  • the relative positional relationship between the target model point and the candidate model point can be used to determine the position information of the target model point in the vehicle coordinate system.
  • the vehicle coordinate system can be established with the target model point as the origin.
  • the coordinate information of the candidate model point in the vehicle coordinate system can be characterized through the relative positional relationship between the target model point and the candidate model point.
  • other model points can also be used as the origin to establish a vehicle coordinate system.
  • the coordinate information of the candidate model point in the vehicle coordinate system can be determined through the relative position relationship between the origin and the candidate model point, and then the relative position between the candidate model point and the target model point can be used relationship, and indirectly determine the coordinate information of the target model point in the vehicle coordinate system.
  • the vehicle coordinate system is established with the target model point as the origin. That is, the coordinate information of the target model point in the vehicle coordinate system is known. Moreover, since the target vehicle is driving on the road at the same time, the world coordinate system can be established with the road as the coordinate plane. Then, the estimated position information of the target model point can be determined by combining the vehicle coordinate system and the road surface coordinate system.
  • the coordinate information of the candidate model point and the candidate world coordinate information are known, and the estimated yaw angle information and the backup Select the world coordinate information to determine the estimated position information of the target model point.
  • the corresponding rotation matrix can be determined by using the estimated yaw angle information, so as to use the rotation matrix and the relative positional relationship between the target model point and the candidate model point to determine the candidate position represented by the candidate world coordinates Relative position information relative to the target model point.
  • the target world coordinate information of the target model point in the road surface coordinate system can be determined according to the two relative positional relationships.
  • the X'O'Y' coordinate system is the vehicle coordinate system established with the model point O' corresponding to the target two-dimensional key point as the coordinate origin, and the XOY coordinate system is the road surface as The road coordinate system established by the coordinate plane, where point O is the corresponding projection point of the model point Q' in the road coordinate system.
  • the candidate model point P is determined, the candidate world coordinate information of the projected point P' of the candidate model point P in the road surface coordinate system can be determined.
  • the vector OP' that is, the relative position information between the coordinate position of the target model point in the road surface coordinate system and the position corresponding to the alternative world coordinates
  • the vector O'P (that is, the relative position relationship between the target model point and the candidate model point) and the estimated yaw angle ⁇
  • the vector O'P' also known as That is, the relative position information of the candidate position represented by the candidate world coordinates relative to the target model point
  • the vector OO' is the difference between the vector OP' and the vector O'P'. Since the alternative world coordinate information of the point P' is known, the coordinate information of the projected point O is calculated, that is, the above estimated position information is obtained. Through this estimation method, the estimated location information can be estimated more simply and accurately.
  • step 204 the server verifies whether the estimated pose satisfies a preset verification condition; and if so, determines the estimated pose as the current actual pose of the target vehicle.
  • step 204 may be the same as or similar to the above step 103, and will not be repeated here.
  • the world coordinates of the target two-dimensional key point in the world coordinate system can be directly determined as estimated position information; and in the image to be processed
  • the step of determining the estimated position information by using the estimated yaw angle information and the relative position relationship between the candidate model point and the target model point makes the estimation process more rational, and to a certain extent Increased confidence in estimated location information.
  • the verification of whether the estimated pose meets the preset verification conditions in the above step 103 or step 204 may include the following sub-steps:
  • Sub-step A determine the projected image coordinates of the matching model point under the estimated pose, the matching model point is the target detected in the vehicle three-dimensional model matched with the target vehicle and the image to be processed The model points corresponding to the two-dimensional key points of the vehicle;
  • the above matching model points may be determined. Specifically, the model points corresponding to the two-dimensional key points detected in the image to be processed may be determined as matching model points. After the matching model points are determined, the projected image coordinates of the matching model points under the estimated pose can be determined.
  • the coordinate information of the matching model point A under the estimated pose is (x w , y w , ⁇ ), and the coordinate information can be substituted into the projection equation to obtain the projected image coordinates (u, v).
  • the projection equation here can be:
  • s is a scale factor, which can be eliminated during calculation; is the internal parameter matrix of the camera, The extrinsic parameter matrix of the camera.
  • z w is known in the road surface coordinate system, x, y can be determined by x w , y w and ⁇ .
  • Sub-step B calculating a reprojection error value based on the projected image coordinates of the matching model points and the image coordinates of the two-dimensional key points detected in the image to be processed;
  • the reprojection error value between the projected image coordinates and the corresponding two-dimensional key point image coordinates can be calculated.
  • the process of calculating the reprojection error value is a well-known technology in the art, and will not be repeated here.
  • Sub-step C judging whether the reprojection error value is smaller than a first error threshold. After the above-mentioned reprojection error value is determined, it can be determined whether the reprojection error value is less than the first error threshold, if less, it can be considered that the determined estimated pose meets the verification condition, and then the estimated pose can be determined as the actual position
  • the first error threshold here may include, for example, values such as 0.1, 0.08, etc., which can substantially represent that the estimated pose is not much different from the actual pose.
  • the pose determination method may further include: if the estimated pose does not satisfy the verification condition, based on the two-dimensional key point information and the three-dimensional vehicle model matched by the target vehicle, The actual pose is determined by using a projection formula; the two-dimensional key point information is obtained by performing two-dimensional key point detection on the target vehicle in the image to be processed.
  • the two-dimensional key point information can be determined using a convolutional neural network such as the one described above. Then the actual pose can be determined by using the two-dimensional key point information, the three-dimensional model of the vehicle and the projection formula. Specifically, the actual pose can be obtained through optimization using the least square method. That is, based on the projection formula, the model points of the 3D vehicle model are projected into the pixel coordinate system, and then the reprojection error value between the projected point corresponding to the model point and the corresponding 2D key point is compared, and the pose is adjusted until the reprojected When the error value meets the requirements or is the smallest, the corresponding pose at this time is determined as the actual pose.
  • the estimated pose does not meet the verification conditions
  • other methods can be used to determine the actual pose, and this method is combined with the estimation method, so that the actual pose can be obtained regardless of whether the estimated pose meets the verification conditions.
  • the vehicle travels straight, most of the cases can meet the verification conditions, and it is only a few cases that need to be determined by other methods.
  • the speed of determining the actual pose is effectively improved by combining a small amount of methods of obtaining the actual pose with a large number of methods of estimating the actual pose using the optimization algorithm such as the above-mentioned least squares method.
  • the historical yaw angle information is determined according to historical images, the historical yaw angle information is in one-to-one correspondence with the historical images, and the shooting time of the historical images is earlier than the The shooting time of the image to be processed, the historical image includes the image of the target vehicle, the historical image and the image to be processed are taken by the same camera; the first historical yaw angle information corresponding to the earliest historical image at the shooting time Determined based on the following steps:
  • Step a performing two-dimensional key point detection on the target vehicle in the earliest historical image at the shooting time, and obtaining the image coordinates of the historical two-dimensional key point;
  • a convolutional neural network to perform key point detection on the earliest captured historical images to obtain image coordinates corresponding to multiple historical two-dimensional key points.
  • Step b determining a historical matching model point according to the historical two-dimensional key point, the historical matching model point is a model point corresponding to the historical two-dimensional key point in the vehicle three-dimensional model matched with the target vehicle;
  • the model point corresponding to the historical two-dimensional key point in the three-dimensional vehicle model can be determined.
  • the determined model point may be determined as the above-mentioned history matching model point.
  • the process of determining the historical matching model point may be similar to the process of the above-mentioned sub-step A, which will not be repeated here.
  • Step c determining an initial estimated pose, and using the initial estimated pose as a current estimated pose
  • Step d using the least squares method to update the current estimated pose and optimize the reprojection error value between the projected image coordinates of the historical matching model points under the current estimated pose and the image coordinates of the historical two-dimensional key points;
  • Step e determining the corresponding yaw angle information when the reprojection error value is less than the second error threshold as the first historical yaw angle information; or determining the corresponding yaw angle information when the current estimated pose update times are greater than the number threshold
  • the angle information is determined as the first historical yaw angle information.
  • the process of determining the first historical yaw angle information is highlighted. In this way, in the images captured after the first historical image is captured, the corresponding The historical yaw angle information is used to determine the corresponding estimated pose later.
  • FIG. 4 shows a structural block diagram of an apparatus for determining a pose provided by an embodiment of the present application.
  • the apparatus for determining a pose may be a module, a program segment, or a code on an electronic device.
  • the device corresponds to the above-mentioned method embodiment in FIG. 1 , and can execute various steps involved in the method embodiment in FIG. 1 .
  • the specific functions of the device can refer to the description above. To avoid repetition, detailed descriptions are appropriately omitted here.
  • the above-mentioned pose determination device may include an acquisition module 401, an estimation module 402, and a verification module 403; wherein, the acquisition module 401 is configured to acquire images to be processed; the images to be processed include images of the target vehicle; Module 402, configured to determine the estimated pose of the target vehicle at the moment when the image to be processed is captured based on the historical yaw angle information of the target vehicle; the historical yaw angle information is the target vehicle The yaw angle information at the time before the time when the image to be processed is captured; the verification module 403 is configured to verify whether the estimated pose satisfies the verification condition; and if so, determine the estimated pose as the The actual pose of the target vehicle at the time when the image to be processed is captured.
  • the acquisition module 401 is configured to acquire images to be processed
  • the images to be processed include images of the target vehicle
  • Module 402 configured to determine the estimated pose of the target vehicle at the moment when the image to be processed is captured based on the historical yaw angle information of the target
  • the estimation module 402 is further configured to: determine the estimated yaw angle information of the target vehicle based on the historical yaw angle information of the target vehicle at a previous moment; Processing the detection results of the two-dimensional key points of the target vehicle in the image to determine estimated position information of the target vehicle.
  • the estimation module 402 is further configured to: if a target two-dimensional key point of the target vehicle is detected in the image to be processed, determine the image coordinates of the target two-dimensional key point; The image coordinates of the target two-dimensional key points and the vehicle three-dimensional model matched with the target vehicle, using the projection formula to determine the target world coordinate information of the target two-dimensional key points in the world coordinate system, and the target world Coordinate information is used as the estimated position information; the world coordinate system includes a coordinate system with the motion plane of the target vehicle as a coordinate plane.
  • the estimation module 402 is further configured to: if no target two-dimensional key point of the target vehicle is detected in the image to be processed, based on the estimated yaw angle information and the The three-dimensional vehicle model matched with the target vehicle is used to determine the estimated position information.
  • the estimation module 402 is further configured to: determine the image coordinates of a candidate two-dimensional key point, the candidate two-dimensional key point is to perform two-dimensional key point detection on the image to be processed, and can At least one of the detected two-dimensional key points; according to the image coordinates of the candidate two-dimensional key points and the three-dimensional model of the vehicle, use a projection formula to determine the candidate two-dimensional key points in the world coordinate system.
  • the world coordinate system includes a coordinate system with the target vehicle’s motion plane as the coordinate plane; according to the relative positional relationship between the target model point and the candidate model point, the estimated yaw angle information and The candidate world coordinate information determines the estimated position information;
  • the target model point is a model point corresponding to the target two-dimensional key point in the vehicle three-dimensional model, and the candidate model point is the candidate model point Select the model points corresponding to the two-dimensional key points in the three-dimensional vehicle model.
  • the verification module 403 is further configured to: determine the projected image coordinates of the matching model point in the estimated pose, the matching model point is the vehicle three-dimensional model matched with the target vehicle and The model point corresponding to the two-dimensional key point of the target vehicle detected in the image to be processed; based on the projected image coordinates of the matching model point and the image coordinates of the two-dimensional key point detected in the image to be processed , calculating a reprojection error value; judging whether the reprojection error value is smaller than a first error threshold.
  • the pose determination device further includes a determination module, and the above determination module is configured to: after verifying whether the estimated pose satisfies the verification condition, if the estimated pose does not meet the Verify the conditions, then based on the two-dimensional key point information, the three-dimensional vehicle model matched by the target vehicle, use the projection formula to determine the actual pose; the two-dimensional key point information is the target in the image to be processed The vehicle is obtained by two-dimensional key point detection.
  • the historical yaw angle information is determined according to historical images, the historical yaw angle information is in one-to-one correspondence with the historical images, and the shooting time of the historical images is earlier than the shooting of the image to be processed
  • the historical image includes the image of the target vehicle, and the historical image and the image to be processed are taken by the same camera;
  • the first historical yaw angle information corresponding to the earliest historical image at the shooting moment is determined based on the following steps: Carry out two-dimensional key point detection to the target vehicle in the historical image with the earliest shooting time, and obtain the image coordinates of the historical two-dimensional key point; determine the historical matching model point according to the historical two-dimensional key point, and the historical matching model point is The model points corresponding to the historical two-dimensional key points in the vehicle three-dimensional model matched with the target vehicle; determine the initial estimated pose, and use the initial estimated pose as the current estimated pose; use the least squares method to update the current Estimating the pose and optimizing the reprojection error value between the projected image coordinates
  • FIG. 5 is a schematic structural diagram of an electronic device for performing a pose determination method provided by an embodiment of the present application.
  • the electronic device may include: at least one processor 501, such as a CPU, and at least one communication interface 502 , at least one memory 503 and at least one communication bus 504 .
  • the communication bus 504 is used to realize the direct connection and communication of these components.
  • the communication interface 502 of the device in the embodiment of the present application is used for signaling or data communication with other node devices.
  • the memory 503 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • the memory 503 may also be at least one storage device located away from the aforementioned processor.
  • Computer-readable instructions are stored in the memory 503 , and when the computer-readable instructions are executed by the processor 501 , the electronic device executes the method process shown in FIG. 1 above.
  • FIG. 5 is only for illustration, and the electronic device may also include more or less components than those shown in FIG. 5 , or have a configuration different from that shown in FIG. 5 .
  • Each component shown in Fig. 5 may be implemented by hardware, software or a combination thereof.
  • An embodiment of the present application provides a readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the method process performed by the electronic device in the method embodiment shown in FIG. 1 is executed.
  • This embodiment discloses a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by the computer, the computer
  • the method includes: acquiring an image to be processed; the image to be processed includes an image of a target vehicle; based on the historical yaw angle information of the target vehicle, determining the target The estimated pose of the vehicle at the time when the image to be processed is captured; the historical yaw angle information is the yaw angle information of the target vehicle at a time before the time when the image to be processed is captured; verify whether the estimated pose is Satisfying the verification condition; and if so, determining the estimated pose as the actual pose of the target vehicle at the time when the image to be processed is captured.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional module in each embodiment of the present application may be integrated to form an independent part, each module may exist independently, or two or more modules may be integrated to form an independent part.
  • the present application provides a pose determining method, electronic equipment and a readable storage medium.
  • the method includes: acquiring an image to be processed; the image to be processed includes an image of a target vehicle; based on historical yaw angle information of the target vehicle, determining an estimated pose of the target vehicle at the time when the image to be processed is taken; The historical yaw angle information is the yaw angle information of the target vehicle at the moment before the image to be processed; verify whether the estimated pose meets the verification condition; and if so, determine the estimated pose is the actual pose of the target vehicle at the time when the image to be processed is captured. In this way, the current actual pose of the target vehicle can be determined based on the determined estimated pose, so that the actual pose of the target vehicle at the shooting moment can be quickly determined.
  • the pose determination method, device, electronic device and readable storage medium of the present application are reproducible and can be used in various industrial applications.
  • the pose determination method, device, electronic device, and readable storage medium of the present application can be applied in the field of intelligent traffic monitoring, such as counting traffic flow, judging whether a driver is driving illegally, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

本公开提供一种位姿确定方法、电子设备及可读存储介质，位姿确定方法包括：获取待处理图像；待处理图像包括目标车辆的图像；基于目标车辆的历史偏航角信息，确定目标车辆在待处理图像拍摄时刻的估计位姿；历史偏航角信息为目标车辆在待处理图像拍摄时刻之前的时刻的偏航角信息；验证估计位姿是否满足验证条件；以及若是，将估计位姿确定为目标车辆在待处理图像拍摄时刻的实际位姿。本公开的位姿确定方法可以快速确定出目标车辆当前对应的实际位姿，增加了系统多路复用的路数。

Description

位姿确定方法、电子设备及可读存储介质
相关申请的交叉引用
本申请要求于2021年08月13日提交中国国家知识产权局的申请号为202110931973.7、名称为“位姿确定方法、装置、电子设备及可读存储介质”的中国专利申请的优先权，上述专利申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理领域，具体而言，涉及一种位姿确定方法、装置、电子设备及可读存储介质。
背景技术
在智能交通监控领域中，准确获得车辆的位姿信息是必要的，确定出的车辆位姿可以应用于例如统计车流量、判断驾驶员是否违规驾驶等场景中。
在确定车辆的位姿时，可以利用车辆的三维模型从采集到的二维图像中恢复出车辆的三维位姿信息。在这过程中，常用的方法是通过最小二乘算法优化车辆的三维投影点和二维图像中的车辆关键点之间的重投影误差，并可以根据重投影误差值最小时对应的位姿信息确定为车辆当前的位姿。这样，计算过程需要花费较多的时间。
发明内容
本申请实施例提供了一种位姿确定方法、电子设备及可读存储介质，用以基于确定的估计位姿确定出目标车辆当前对应的实际位姿，可以快速确定出目标车辆在拍摄时刻对应的实际位姿。
本申请的一些实施例提供了一种位姿确定方法，该方法可以包括：获取待处理图像；所述待处理图像包括目标车辆的图像；基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿；所述历史偏航角信息为所述目标车辆在所述待处理图像拍摄时刻之前的时刻的偏航角信息；验证所述估计位姿是否满足验证条件；以及若是，将所述估计位姿确定为所述目标车辆在所述待处理图像拍摄时刻的实际位姿。这样，可以缩短确定实际位姿的计算过程，增加了系统多路复用的路数。
可选地，所述估计位姿可以包括估计偏航角信息和估计位置信息；以及所述基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿，可以包括：基于所述目标车辆在上一时刻的历史偏航角信息，确定所述目标车辆的估计偏航角信息；和/或基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息。这样，基于目标车辆在短时间内运动方向不变的情况下，可以基于将上一时刻的历史偏航角信息确定出估计偏航角信息，估计过程较为合理。
可选地，所述基于所述目标车辆在上一时刻的历史偏航角信息，确定所述目标车辆的估计偏航角信息，可以包括：将所述上一时刻的历史偏航角信息确定为所述估计偏航角信息；或者将在所述上一时刻的历史偏航角的基础上补偿目标角度之后得到的偏航角确定为所述估计偏航角信息。
可选地，所述目标角度可以是基于摄像机拍摄图像的频率以及目标车辆的车速确定的。
可选地，所述基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息，可以包括：若在所述待处理图像中检测到所述目标车辆的目标二维关键点，确定所述目标二维关键点的图像坐标；根据所述目标二维关键点的图像坐标以及与所述目标车辆匹配的车辆三维模型，利用投影公式确定所述目标二维关键点在世界坐标系下的目标世界坐标信息，并将所述目标世界坐标信息作为所述估计位置信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系。这样，基于目标车辆在短时间内运动方向不变的情况，可以直接基于检测到的目标二维关键点的图像坐标确定为估计位置信息，估计过程简便直观。
可选地，所述目标二维关键点可以包括与所述目标车辆上的前车标或者左视镜所对应的关键点。
可选地，所述基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息，可以包括：若在所述待处理图像中未检测到所述目标车辆的目标二维关键点，则基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息。这样，在未检测到目标二维关键点时，也可以确定出估计位置信息。
可选地，所述基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息，可以包括：确定备选二维关键点的图像坐标，所述备选二维关键点为对所述待处理图像进行二维关键点检测，能够检测到的二维关键点中的至少一个；根据所述备选二维关键点的图像坐标以及所述车辆三维模型，利用投影公式确定所述备选二维关键点在世界坐标系下的备选世界坐标信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系；根据目标模型点与备选模型点之间的相对位置关系、所述估计偏航角信息以及所述备选世界坐标信息，确定所述估计位置信息；所述目标模型点为所述目标二维关键点在所述车辆三维模型中对应的模型点，所述备选模型点为所述备选二维关键点在所述车辆三维模型中对应的模型点。这样，可以间接获得估计位置信息。
可选地，可以通过下述方式确定所述备选二维关键点：利用卷积神经网络对所述待处理图像进行二维关键点检测，得到各二维关键点的图像坐标和置信度；将取置信度最大的二维关键点作为所述备选二维关键点。
可选地，所述验证所述估计位姿是否满足验证条件，可以包括：确定匹配模型点在所述估计位姿下的投影图像坐标，所述匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述待处理图像中检测出来的所述目标车辆的二维关键点对应的模型点；基于所述匹配模型点的投影图像坐标与所述待处理图像中检测出来的二维关键点的图像坐标，计算重投影误差值；判断所述重投影误差值是否小于第一误差阈值。这样，使得满足验证条件的估计位姿与实际位姿更加贴合。
可选地，在所述验证所述估计位姿是否满足验证条件之后，所述方法还可以包括：若所述估计位姿未满足所述验证条件，则基于二维关键点信息、所述目标车辆匹配的车辆三维模型，利用投影公式确定所述实际位姿；所述二维关键点信息是对所述待处理图像中的所述目标车辆进行二维关键点检测得到的。这样，当估计位姿不满足验证条件时，可以利用其他方式确定出实际位姿，并且该方式与估计方式结合，使得不论估计位姿是否满足验证条件，均可得到实际位姿。
可选地，所述历史偏航角信息可以是根据历史图像确定的，所述历史偏航角信息与所述历史图像一一对应，所述历史图像的拍摄时刻早于所述待处理图像的拍摄时刻，所述历史图像包括所述目标车辆的图像，所述历史图像和所述待处理图像由同一相机拍摄；对应于拍摄时刻最早的历史图像的首个历史偏航角信息可以基于以下步骤确定：对拍摄时刻最早的历史图像中的目标车辆进行二维关键点检测，得到历史二维关键点的图像坐标；根据所述历史二维关键点，确定历史匹配模型点，所述历史匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述历史二维关键点对应的模型点；确定初始估计位姿，并将所述初始估计位姿作为当前估计位姿；利用最小二乘法更新当前估计位姿并优化所述历史匹配模型点在当前估计位姿下的投影图像坐标与所述历史二维关键点的图像坐标之间的重投影误差值；将所述重投影误差值小于第二误差阈值时对应的偏航角信息确定为所述首个历史偏航角信息；或者将当前估计位姿更新次数大于次数阈值时对应的偏航角信息确定为所述首个历史偏航角信息。以此提供一种可以确定出首个偏航角的方式。
本申请的又一些实施例还提供了一种电子设备，该电子设备可以包括处理器以及存储器，所述存储器存储有计算机可读取指令，当所述计算机可读取指令由所述处理器执行时，运行如上述本申请的一些实施例提供的所述方法中的步骤。
本申请的再一些实施例还提供了一种可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时运行如上述本申请的一些实施例提供的所述方法中的步骤。
本申请的再一些实施例还提供了一种计算机程序产品，该计算机程序产品可以包括计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，所述计算机能够执行如上述本申请的一些实施例提供的所述方法中的步骤。
本申请的其他特征和优点将在随后的说明书阐述，并且，部分地从说明书中变得显而易见，或者通过实施本申请实施例了解。本申请的目的和其他优点可通过在所写的说明书、权利要求书、以及附图中所特别指出的结构来实现和获得。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本申请的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1为本申请实施例提供的一种位姿确定方法的流程图；
图2为本申请实施例提供的另一种位姿确定方法的流程图；
图3为本申请涉及的确定估计位姿的一种应用场景的示意图；
图4为本申请实施例提供的一种位姿确定装置的结构框图；
图5为本申请实施例提供的一种用于执行位姿确定方法的电子设备的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中附图，对本申请实施例中的技术方案进行清楚、完整地描述。显然，所描述的实施例是本本申请的一部分实施例，而不是全部的实施例。因此，以下对在附图中提供的本申请的实施例的详细描述并非旨在限制要求保护的本申请的范围，而是仅仅表示本申请的选定实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
近年来，基于人工智能的计算机视觉、深度学习、机器学习、图像处理、图像识别等技术研究取得了重要进展。人工智能(Artificial Intelligence，AI)是研究、开发用于模拟、延伸人的智能的理论、方法、技术及应用系统的新兴科学技术。人工智能学科是一门综合性学科，涉及芯片、大数据、云计算、物联网、分布式存储、深度学习、机器学习、神经网络等诸多技术种类。计算机视觉作为人工智能的一个重要分支，具体是让机器识别世界，计算机视觉技术通常包括人脸识别、活体检测、指纹识别与防伪验证、生物特征识别、人脸检测、行人检测、目标检测、行人识别、图像处理、图像识别、图像语义理解、图像检索、文字识别、视频处理、视频内容识别、三维重建、虚拟现实、增强现实、同步定位与地图构建(SLAM)、计算摄影、机器人导航与定位等技术。随着人工智能技术的研究和进步，该项技术在众多领域展开了应用，例如安防、城市管理、交通管理、楼宇管理、园区管理、人脸通行、人脸考勤、物流管理、仓储管理、机器人、智能营销、计算摄影、手机影像、云服务、智能家居、穿戴设备、无人驾驶、自动驾驶、智能医疗、人脸支付、 人脸解锁、指纹解锁、人证核验、智慧屏、智能电视、摄像机、移动互联网、网络直播、美颜、美妆、医疗美容、智能测温等领域。
相关技术中存在确定位姿信息花费时间长，不能充分满足系统在实现多路复用技术时的信号路数要求的问题。为了解决上述技术问题，本申请提供一种位姿确定方法、装置、电子设备及可读存储介质。利用车辆在短时间内运动方向变化不大的特性，通过将目标车辆的历史偏航角信息确定为目标车辆在拍摄时刻的估计位姿，并通过预先设置的验证条件对估计位姿进行验证的技术方案，可以快速确定出目标车辆在拍摄时刻对应的实际位姿，增加了系统多路复用的路数。需要说明的是，上述位姿确定方法可以应用于服务器、云平台等实质上可以提供计算、信息传输以及存储功能的数据处理中心。示例性地，本申请在后文中以服务器为例，具体介绍上述位姿确定方法。
请参考图1，其示出了本申请实施例提供的一种位姿确定方法的流程图。如图1所示，该位姿确定方法可以包括以下步骤101至步骤103。
步骤101，服务器获取待处理图像；所述待处理图像包括目标车辆的图像；
上述目标车辆例如可以包括货车、面包车、小轿车等多种车型。
在一些应用场景中，服务器可以获取上述待处理图像。上述待处理图像例如可以是从视频中截取的包括目标车辆的图像，也可以是通过相机拍摄的包括目标车辆的图像。
步骤102，服务器基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿；所述历史偏航角信息为所述目标车辆在所述待处理图像拍摄时刻之前的时刻的偏航角信息；
可选地，位姿可以包括位置和姿态。可选地，姿态可以用偏航角表示。
当获取到待处理图像之后，可以获取目标车辆的历史偏航角信息。这里，历史偏航角信息是根据待处理图像的拍摄时刻之前拍摄的图像确定出来的目标车辆的偏航角信息。具体的，偏航角信息可以和图像的拍摄时刻一一对应，比如一张图像可以确定出来一个对应于唯一拍摄时刻的偏航角。历史偏航角信息可以包括在待处理图像之前拍摄的一张或多张图像中确定出来的一个或多个偏航角信息。
在一些应用场景中，例如可以通过最小二乘法优化车辆三维模型的投影点和待处理图像中的二维关键点之间的重投影误差，确定目标车辆的第一个偏航角，也就得到了目标车辆的第一个偏航角信息。估计偏航角信息可以作为第二个偏航角信息以及之后确定的偏航角信息对应的历史偏航角信息。这里，第二个偏航角信息以及之后确定的偏航角信息分别与拍摄的第二张图像以及之后拍摄的图像对应。通过最小二乘法确定目标车辆的首个历史偏航角的过程在后文的相关部分有具体介绍，此处不赘述。在另一些应用场景中，例如也可以将预先设定的偏航角初始值确定为历史偏航角信息。这里，可以根据实际情况设定偏 航角初始值，例如5°、30°等。
确定了历史偏航角信息之后，可以确定目标车辆在待处理图像拍摄时刻的估计位姿。上述估计位姿例如可以包括目标车辆的位置信息和偏航角信息。上述位置信息例如可以利用目标车辆在世界坐标系下的坐标信息表征。这样，上述估计位姿例如可以包括目标车辆在世界坐标系(一般为以路面为坐标面建立的大地坐标系)下的坐标信息和偏航角信息，其例如可以坐标(X，Y，θ)表征。这里的X，Y可以表征任意的在世界坐标系范围内的坐标数值，θ可以表征任意的在(0,2π)范围内的度数。
步骤103，服务器验证所述估计位姿是否满足验证条件；以及若是，将所述估计位姿确定为所述目标车辆在所述待处理图像拍摄时刻的实际位姿。
当确定了目标车辆的估计位姿之后，可以验证该估计位姿是否满足验证条件。
当确定了估计位姿满足预设验证条件之后，可以将估计位姿确定为目标车辆当前对应的实际位姿。例如，验证了目标车辆的估计位姿(4，5，30°)满足验证条件之后，可以将(4，5，30°)确定为目标车辆当前对应的实际位姿。
实践中，车辆在道路上行驶时，除了少数车辆需要变道行驶之外，大部分车辆都会按照车道行驶，并且这些车辆一般处于直线行驶状态。也即，车辆的偏航角在短时间内变化不大，因此，可以通过上述步骤101至步骤103，基于目标车辆的历史偏航角信息估计出目标车辆的实际位姿，继而可以缩短确定实际位姿的计算过程，增加了系统多路复用的路数。
请参考图2，图2示出了本申请实施例提供的另一种位姿确定方法的流程图。如图2所示，位姿确定方法可以包括以下步骤201至步骤204。
步骤201，服务器获取待处理图像；所述待处理图像包括目标车辆的图像；
上述步骤201的具体实现过程以及取得的技术效果可以与上述步骤101相同或相似，此处不赘述。
步骤202，服务器基于所述目标车辆在上一时刻的历史偏航角信息，确定所述目标车辆的估计偏航角信息；
实践中，车辆在道路上行驶时，除了少数需要变道行驶之外，都是按照当前的车道行驶，偏航角在短时间内(例如1秒、3秒、20秒等)不会发生较大改变。例如，当车辆在较直的车道上以匀速或者加速行驶时，在一定时间内，其偏航角都不会发生较大变化；当车辆在弯曲道路上行驶时，当间隔时间足够短时，其偏航角也不会发生较大变化。
因此，可以在确定了目标车辆在上一时刻的历史偏航角信息之后，基于该历史偏航角信息估计该目标车辆在当前时刻的估计偏航角信息。
在一些可选的实现方式中，上述步骤202可以包括：将所述上一时刻的历史偏航角信 息确定为所述估计偏航角信息。
当目标车辆不更改车道时，可以粗略认为该目标车辆在短时间内处于直线行驶状态，继而，可以将其在上一时刻的历史偏航角确定为当前时刻的估计偏航角，以简化估计偏航角信息的估计过程。
在另一些可选的实现方式中，上述步骤202可以包括：将在所述上一时刻的历史偏航角的基础上补偿一定角度之后得到的偏航角确定为所述估计偏航角信息。
例如，目标车辆在角度为“xx”的弯道上行驶时，可以通过摄像机拍摄图像的频率以及目标车辆的车速估计出补偿的角度，继而可以将该补偿的角度添加到上一时刻的历史偏航角中，以将此时得到的偏航角信息确定为估计偏航角信息。这里，上述“xx”可以为在合理范围的诸如“25°”、“15°”等。
步骤203，服务器基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息。
在确定目标车辆的估计位置信息时，可以基于在所述待处理图像中对二维关键点的检测结果进行确定。这里的二维关键点例如可以包括与目标车辆上的前车标、目标车辆的左视镜等对应的关键点。
在一些应用场景中，例如可以利用卷积神经网络、热图等检测二维关键点。
在一些可选的实现方式中上述步骤203可以包括步骤2031：若在所述待处理图像中检测到所述目标车辆的目标二维关键点，确定所述目标二维关键点的图像坐标；根据所述目标二维关键点的图像坐标以及与所述目标车辆匹配的车辆三维模型，利用投影公式确定所述目标二维关键点在世界坐标系下的目标世界坐标信息，并将所述目标世界坐标信息作为所述估计位置信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系。
在一些应用场景中，当在上述待处理图像中检测到目标二维关键点时，可以利用诸如上述的卷积神经网络确定出该目标二维关键点在待处理图像中的图像坐标，然后可以基于该图像坐标，利用投影公式以及车辆三维模型确定出目标二维关键点在世界坐标系下的目标世界坐标信息。这里的车辆三维模型例如可以通过识别待处理图像的目标车辆之后，在预设模型库中匹配得到。上述预设模型库例如可以包括诸如面包车、小轿车、货车等分别对应的车辆三维模型。在一些应用场景中，可以通过识别待处理图像中的目标车辆得到与该目标车辆对应的车辆三维模型。在这些应用场景中，通过车辆三维模型可以确定出目标车辆的长度、宽度、高度以及各个模型点之间的相对位置关系等信息。因此，车辆三维模型中各个模型点的位置例如可以目标二维关键点对应的模型点为参照点进行表征，也可以利用其它模型点为参照点进行表征。
例如，在确定了目标二维关键点对应的图像坐标为(u，v)之后，可以基于已知的车 辆三维模型确定出目标车辆的高度信息，然后再利用投影公式，即可计算出目标车辆在世界坐标系下的目标世界坐标信息为(x w，y w，z w)。在这些应用场景中，上述u、v、x w、y w、z w可以表征在其所属坐标系下的任意数字，其中z w已知。上述投影公式例如可以包括：
Figure PCTCN2022108678-appb-000001
其中，s为尺度因子，
Figure PCTCN2022108678-appb-000002
为相机的内参数矩阵，
Figure PCTCN2022108678-appb-000003
为相机的外参数矩阵。这里，需要说明的是：世界坐标系以目标车辆的运动平面(例如路面)为坐标面，在目标车辆的车辆三维模型已知的情况下，可以将目标二维关键点在世界坐标系下的高度信息视为已知(也即，z w已知)。这样，利用上述投影公式进行计算时，可以得到具有唯一解的二元二次方程。继而可以得到唯一的目标世界坐标信息。
在一些可选的实现方式中，上述步骤203可以包括步骤2032：若在所述待处理图像中未检测到所述目标车辆的目标二维关键点，则基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息。
当在待处理图像中没有检测到目标二维关键点时，可以基于估计偏航角信息以及车辆三维模型，确定估计位置信息。例如，当确定的目标二维关键点为目标车辆的左视镜时，如果待处理图像是基于摄像机位于目标车辆的右前方拍摄得到，此时若在待处理图像中检测不到左视镜，则当前的检测结果为未在待处理图像中检测到目标二维关键点。继而可以通过估计偏航角和车辆三维模型估计估计位置信息。
在一些可选的实现方式中，所述步骤2032中的基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息，可以包括以下子步骤：
子步骤1，确定备选二维关键点的图像坐标，所述备选二维关键点为对所述待处理图像进行二维关键点检测，能够检测到的二维关键点中的至少一个；
在对二维关键点进行检测时，可以检测到多个二维关键点，此时，可以将检测到的其中一个二维关键点确定为备选二维关键点。继而，可以确定出该备选二维关键的图像坐标。
可选地，可以利用卷积神经网络对待处理图像进行二维关键点检测，从而可以得到各二维关键点的图像坐标和置信度。此时，可以选取置信度最大的二维关键点作为备选二维关键点，在一定程度上也可以提高估计位置信息的置信度。
子步骤2，根据所述备选二维关键点的图像坐标以及所述车辆三维模型，利用投影公式确定所述备选二维关键点在世界坐标系下的备选世界坐标信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系；
上述子步骤2的实现过程可以与上述步骤2031的实现过程相似，此处不赘述。
子步骤3，根据目标模型点与备选模型点之间的相对位置关系、所述估计偏航角信息以及所述备选世界坐标信息，确定所述估计位置信息；所述目标模型点为所述目标二维关键点在所述车辆三维模型中对应的模型点，所述备选模型点为所述备选二维关键点在所述车辆三维模型中对应的模型点。
可选地，可以利用目标模型点与备选模型点之间的相对位置关系，确定出目标模型点在车辆坐标系下的位置信息。在一些应用场景中，可以目标模型点为原点建立车辆坐标系。此时，通过目标模型点与备选模型点之间的相对位置关系可以表征出备选模型点在该车辆坐标系下的坐标信息。在另一些应用场景中，也可以其他模型点为原点建立车辆坐标系。此时，可以通过该原点与备选模型点之间的相对位置关系确定出备选模型点在该车辆坐标系下的坐标信息，然后可以利用备选模型点与目标模型点之间的相对位置关系，间接确定出目标模型点在该车辆坐标系下的坐标信息。
在本实施例中，以目标模型点为原点建立车辆坐标系。也即，目标模型点在车辆坐标系下的坐标信息已知。并且，由于目标车辆同时行驶于路面上，因此可以路面为坐标面建立世界坐标系。继而，可以联立车辆坐标系以及路面坐标系确定出目标模型点的估计位置信息。
在以目标模型点为原点建立的车辆坐标系以及以路面为坐标面建立的路面坐标系中，已知备选模型点的坐标信息以及备选世界坐标信息，可以利用估计偏航角信息以及备选世界坐标信息确定出目标模型点的估计位置信息。具体的，可以利用估计偏航角信息确定出对应的旋转矩阵，以利用该旋转矩阵以及目标模型点与备选模型点之间的相对位置关系，确定出备选世界坐标所表征的备选位置相对于目标模型点的相对位置信息。此时，由于目标模型点与备选模型点之间的相对位置关系已知，因此，可以根据这两个相对位置关系，确定出目标模型点在路面坐标系下的目标世界坐标信息。
例如，在图3所示应用场景的示意图中，X＇O＇Y＇坐标系为以目标二维关键点对应的 模型点O＇为坐标原点建立的车辆坐标系，XOY坐标系为以路面为坐标面建立的路面坐标系，其中，点O为模型点Q＇在路面坐标系下对应的投影点。当确定了备选模型点P之后，可以确定出该备选模型点P在路面坐标系下的投影点P＇的备选世界坐标信息。此时，可以得到向量OP＇(也即，目标模型点在路面坐标系下的坐标位置与备选世界坐标对应的位置之间的相对位置信息)。基于向量O＇P(也即，目标模型点与备选模型点之间的相对位置关系)以及估计偏航角θ，通过旋转矩阵R(θ)，即可计算出向量O＇P＇(也即，备选世界坐标所表征的备选位置相对于目标模型点的相对位置信息)，则向量OO＇为向量OP＇与向量O＇P＇之差。而由于点P＇的备选世界坐标信息已知，便计算得到了投影点O的坐标信息，也即，得到了上述估计位置信息。通过这种估计方法，可以较为简便准确地估计到估计位置信息。
步骤204，服务器验证所述估计位姿是否满足预设验证条件；以及若是，将所述估计位姿确定为所述目标车辆当前对应的实际位姿。
上述步骤204的具体实现过程以及取得的技术效果可以与上述步骤103相同或相似，此处不赘述。
在本实施例中，突出了在待处理图像中检测到目标二维关键点时，可以直接将该目标二维关键点在世界坐标系下的世界坐标确定为估计位置信息；以及在待处理图像中未检测到目标二维关键点时，利用估计偏航角信息以及备选模型点与目标模型点之间的相对位置关系确定出估计位置信息的步骤，使得估计过程更加合理化，在一定程度上提高了估计位置信息的置信度。
在一些可选的实现方式中，上述步骤103或步骤204中的所述验证所述估计位姿是否满足预设验证条件，可以包括以下子步骤：
子步骤A，确定匹配模型点在所述估计位姿下的投影图像坐标，所述匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述待处理图像中检测出来的所述目标车辆的二维关键点对应的模型点；
在一些应用场景中，可以确定上述匹配模型点。具体的，可以将在待处理图像中检测出的二维关键点所对应的模型点确定为匹配模型点。确定了匹配模型点之后，可以确定出匹配模型点在估计位姿下的投影图像坐标。
例如，已知匹配模型点A在估计位姿下的坐标信息为(x w，y w,θ)，可以将该坐标信息代入投影方程中，以得到投影图像坐标(u，v)。这里的投影方程可以为：
Figure PCTCN2022108678-appb-000004
；其中，s为尺度因子，其可以在计算过程中消除；
Figure PCTCN2022108678-appb-000005
为相机的内参数矩阵，
Figure PCTCN2022108678-appb-000006
相机的外参数矩阵。z w在路面坐标系下已知，x，y可以通过x w、y w以及θ确定。
子步骤B，基于所述匹配模型点的投影图像坐标与所述待处理图像中检测出来的二维关键点的图像坐标，计算重投影误差值；
确定出匹配模型点的投影图像坐标之后，可以根据投影图像坐标与其对应的二维关键点的图像坐标，计算两者之间的重投影误差值。这里，计算重投影误差值的过程为本领域公知技术，此处不赘述。
子步骤C，判断所述重投影误差值是否小于第一误差阈值。当确定了上述重投影误差值之后，可以确定该重投影误差值是否小于第一误差阈值，如果小于，可以视为确定的估计位姿满足验证条件，继而可以将该估计位姿确定为实际位姿这里的第一误差阈值例如可以包括0.1、0.08等实质上可以表征估计位姿与实际位姿相差不大的数值。
在一些可选的实现方式中，所述位姿确定方法还可以包括：若所述估计位姿未满足所述验证条件，则基于二维关键点信息、所述目标车辆匹配的车辆三维模型，利用投影公式确定所述实际位姿；所述二维关键点信息是对所述待处理图像中的所述目标车辆进行二维关键点检测得到的。
当估计的估计位姿不满足验证条件时，可以利用诸如上述的卷积神经网络确定出二维关键点信息。然后可以利用二维关键点信息、车辆三维模型以及投影公式，确定出实际位姿。具体的，可以利用最小二乘法优化得到实际位姿。也即，基于投影公式，将车辆三维模型的模型点投影到像素坐标系中，然后比较模型点对应的投影点与对应二维关键点之间的重投影误差值，调整位姿直至将重投影误差值满足要求或最小时，将此时对应的位姿确定为实际位姿。
当估计的估计位姿不满足验证条件时，可以利用其他方式确定出实际位姿，并且该方式与估计方式结合，使得不论估计位姿是否满足验证条件，均可得到实际位姿。而由于车辆直线行驶，大部分情况都能满足验证条件，需要用其他方式确定的是少数情况。这样，通过少量利用诸如上述最小二乘法优化算法得到实际位姿的方式与大量的估计实际位姿的方式结合，有效提升了确定实际位姿的速度。
在一些可选的实现方式中，所述历史偏航角信息是根据历史图像确定的，所述历史偏航角信息与所述历史图像一一对应，所述历史图像的拍摄时刻早于所述待处理图像的拍摄时刻，所述历史图像包括所述目标车辆的图像，所述历史图像和所述待处理图像由同一相机拍摄；对应于拍摄时刻最早的历史图像的首个历史偏航角信息基于以下步骤确定：
步骤a，对拍摄时刻最早的历史图像中的目标车辆进行二维关键点检测，得到历史二维关键点的图像坐标；
在一些应用场景中，可以利用诸如卷积神经网络对最早拍摄得到的历史图像进行关键点检测，得到多个历史二维关键点对应的图像坐标。
步骤b，根据所述历史二维关键点，确定历史匹配模型点，所述历史匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述历史二维关键点对应的模型点；
检测得到历史二维关键点之后，可以确定车辆三维模型中与该历史二维关键点对应的模型点。并可以将确定出的模型点确定为上述历史匹配模型点。这里，确定历史匹配模型点的过程可以与上述子步骤A的过程相似，此处不赘述。
步骤c，确定初始估计位姿，并将所述初始估计位姿作为当前估计位姿；
步骤d，利用最小二乘法更新当前估计位姿并优化所述历史匹配模型点在当前估计位姿下的投影图像坐标与所述历史二维关键点的图像坐标之间的重投影误差值；
步骤e，将所述重投影误差值小于第二误差阈值时对应的偏航角信息确定为所述首个历史偏航角信息；或者将当前估计位姿更新次数大于次数阈值时对应的偏航角信息确定为所述首个历史偏航角信息。
通过上述步骤a至步骤e，突出了确定出首个历史偏航角信息的过程，这样，在拍摄得到首个历史图像之后拍摄得到的图像中，可以利用前一拍摄得到的图像确定出对应的历史偏航角信息，以利于后期确定出对应的估计位姿。
请参考图4，其示出了本申请实施例提供的一种位姿确定装置的结构框图，该位姿确定装置可以是电子设备上的模块、程序段或代码。应理解，该装置与上述图1方法实施例对应，能够执行图1方法实施例涉及的各个步骤，该装置具体的功能可以参见上文中的描述，为避免重复，此处适当省略详细描述。
可选地，上述位姿确定装置可以包括获取模块401、估计模块402和验证模块403；其 中，获取模块401被配置成用于获取待处理图像；所述待处理图像包括目标车辆的图像；估计模块402，被配置成用于基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿；所述历史偏航角信息为所述目标车辆在所述待处理图像拍摄时刻之前的时刻的偏航角信息；验证模块403，被配置成用于验证所述估计位姿是否满足验证条件；以及若是，将所述估计位姿确定为所述目标车辆在所述待处理图像拍摄时刻的实际位姿。
可选地，估计模块402还被配置成用于：基于所述目标车辆在上一时刻的历史偏航角信息，确定所述目标车辆的估计偏航角信息；和/或基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息。
可选地，所述估计模块402还被配置成用于：若在所述待处理图像中检测到所述目标车辆的目标二维关键点，确定所述目标二维关键点的图像坐标；根据所述目标二维关键点的图像坐标以及与所述目标车辆匹配的车辆三维模型，利用投影公式确定所述目标二维关键点在世界坐标系下的目标世界坐标信息，并将所述目标世界坐标信息作为所述估计位置信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系。
可选地，所述估计模块402还被配置成用于：若在所述待处理图像中未检测到所述目标车辆的目标二维关键点，则基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息。
可选地，所述估计模块402还被配置成用于：确定备选二维关键点的图像坐标，所述备选二维关键点为对所述待处理图像进行二维关键点检测，能够检测到的二维关键点中的至少一个；根据所述备选二维关键点的图像坐标以及所述车辆三维模型，利用投影公式确定所述备选二维关键点在世界坐标系下的备选世界坐标信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系；根据目标模型点与备选模型点之间的相对位置关系、所述估计偏航角信息以及所述备选世界坐标信息，确定所述估计位置信息；所述目标模型点为所述目标二维关键点在所述车辆三维模型中对应的模型点，所述备选模型点为所述备选二维关键点在所述车辆三维模型中对应的模型点。
可选地，所述验证模块403还被配置成用于：确定匹配模型点在所述估计位姿下的投影图像坐标，所述匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述待处理图像中检测出来的所述目标车辆的二维关键点对应的模型点；基于所述匹配模型点的投影图像坐标与所述待处理图像中检测出来的二维关键点的图像坐标，计算重投影误差值；判断所述重投影误差值是否小于第一误差阈值。
可选地，所述位姿确定装置还包括确定模块，以及上述确定模块被配置成用于：在所述验证所述估计位姿是否满足验证条件之后，若所述估计位姿未满足所述验证条件，则基 于二维关键点信息、所述目标车辆匹配的车辆三维模型，利用投影公式确定所述实际位姿；所述二维关键点信息是对所述待处理图像中的所述目标车辆进行二维关键点检测得到的。
可选地，所述历史偏航角信息是根据历史图像确定的，所述历史偏航角信息与所述历史图像一一对应，所述历史图像的拍摄时刻早于所述待处理图像的拍摄时刻，所述历史图像包括所述目标车辆的图像，所述历史图像和所述待处理图像由同一相机拍摄；对应于拍摄时刻最早的历史图像的首个历史偏航角信息基于以下步骤确定：对拍摄时刻最早的历史图像中的目标车辆进行二维关键点检测，得到历史二维关键点的图像坐标；根据所述历史二维关键点，确定历史匹配模型点，所述历史匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述历史二维关键点对应的模型点；确定初始估计位姿，并将所述初始估计位姿作为当前估计位姿；利用最小二乘法更新当前估计位姿并优化所述历史匹配模型点在当前估计位姿下的投影图像坐标与所述历史二维关键点的图像坐标之间的重投影误差值；将所述重投影误差值小于第二误差阈值时对应的偏航角信息确定为所述首个历史偏航角信息；或者将当前估计位姿更新次数大于次数阈值时对应的偏航角信息确定为所述首个历史偏航角信息。
需要说明的是，本领域技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再重复描述。
请参照图5，图5为本申请实施例提供的一种用于执行位姿确定方法的电子设备的结构示意图，所述电子设备可以包括：至少一个处理器501，例如CPU，至少一个通信接口502，至少一个存储器503和至少一个通信总线504。其中，通信总线504用于实现这些组件直接的连接通信。其中，本申请实施例中设备的通信接口502用于与其他节点设备进行信令或数据的通信。存储器503可以是高速RAM存储器，也可以是非易失性的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器503可选的还可以是至少一个位于远离前述处理器的存储装置。存储器503中存储有计算机可读取指令，当所述计算机可读取指令由所述处理器501执行时，电子设备执行如上述图1所示方法过程。
可以理解，图5所示的结构仅为示意，所述电子设备还可包括比图5中所示更多或者更少的组件，或者具有与图5所示不同的配置。图5中所示的各组件可以采用硬件、软件或其组合实现。
本申请实施例提供一种可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时，执行如图1所示方法实施例中电子设备所执行的方法过程。
本实施例公开一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执 行时，计算机能够执行上述各方法实施例所提供的方法，例如，该方法包括：获取待处理图像；所述待处理图像包括目标车辆的图像；基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿；所述历史偏航角信息为所述目标车辆在所述待处理图像拍摄时刻之前的时刻的偏航角信息；验证所述估计位姿是否满足验证条件；以及若是，将所述估计位姿确定为所述目标车辆在所述待处理图像拍摄时刻的实际位姿。
在本申请所提供的实施例中，应该理解到，所揭露装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
另外，作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
再者，在本申请各个实施例中的各功能模块可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或两个以上模块集成形成一个独立的部分。
在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。
以上所述仅为本申请的实施例而已，并不用于限制本申请的保护范围，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。
工业实用性
本申请提供了一种位姿确定方法、电子设备及可读存储介质。该方法包括：获取待处理图像；所述待处理图像包括目标车辆的图像；基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿；所述历史偏航角信息为所述目标车辆在所述待处理图像拍摄时刻之前的时刻的偏航角信息；验证所述估计位姿是否满足验证条件；以及若是，将所述估计位姿确定为所述目标车辆在所述待处理图像拍摄时刻的实际位姿。这样，可以基于确定的估计位姿确定出目标车辆当前对应的实际位姿，从而可以快速确定出目标车辆在拍摄时刻对应的实际位姿。
此外，可以理解的是，本申请的位姿确定方法、装置、电子设备及可读存储介质是可 以重现的，并且可以用在多种工业应用中。例如，本申请的位姿确定方法、装置、电子设备及可读存储介质可以在智能交通监控领域中应用于例如统计车流量、判断驾驶员是否违规驾驶等场景中。

Claims (15)

  1. 一种位姿确定方法，其特征在于，包括：
    获取待处理图像；所述待处理图像包括目标车辆的图像；
    基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿；所述历史偏航角信息为所述目标车辆在所述待处理图像拍摄时刻之前的时刻的偏航角信息；
    验证所述估计位姿是否满足验证条件；以及
    若是，将所述估计位姿确定为所述目标车辆在所述待处理图像拍摄时刻的实际位姿。
  2. 根据权利要求1所述的方法，其特征在于，所述估计位姿包括估计偏航角信息和估计位置信息；以及
    所述基于所述目标车辆的历史偏航角信息，确定所述目标车辆在所述待处理图像拍摄时刻的估计位姿，包括：
    基于所述目标车辆在上一时刻的历史偏航角信息，确定所述目标车辆的估计偏航角信息；和/或
    基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息。
  3. 根据权利要求2所述的方法，其特征在于，所述基于所述目标车辆在上一时刻的历史偏航角信息，确定所述目标车辆的估计偏航角信息，包括：
    将所述上一时刻的历史偏航角信息确定为所述估计偏航角信息；或者
    将在所述上一时刻的历史偏航角的基础上补偿目标角度之后得到的偏航角确定为所述估计偏航角信息。
  4. 根据权利要求3所述的方法，其特征在于，所述目标角度是基于摄像机拍摄图像的频率以及所述目标车辆的车速确定的。
  5. 根据权利要求2至4中任一项所述的方法，其特征在于，所述基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息，包括：
    若在所述待处理图像中检测到所述目标车辆的目标二维关键点，确定所述目标二维关键点的图像坐标；
    根据所述目标二维关键点的图像坐标以及与所述目标车辆匹配的车辆三维模型，利用投影公式确定所述目标二维关键点在世界坐标系下的目标世界坐标信息，并将所述目标世界坐标信息作为所述估计位置信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系。
  6. 根据权利要求5所述的方法，其特征在于，所述目标二维关键点包括与所述目标车辆上的前车标或者左视镜所对应的关键点。
  7. 根据权利要求2至6中任一项所述的方法，其特征在于，所述基于在所述待处理图像中对所述目标车辆的二维关键点的检测结果，确定所述目标车辆的估计位置信息，包括：
    若在所述待处理图像中未检测到所述目标车辆的目标二维关键点，则基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息。
  8. 根据权利要求7所述的方法，其特征在于，所述基于所述估计偏航角信息以及与所述目标车辆匹配的车辆三维模型，确定所述估计位置信息，包括：
    确定备选二维关键点的图像坐标，所述备选二维关键点为对所述待处理图像进行二维关键点检测，能够检测到的二维关键点中的至少一个；
    根据所述备选二维关键点的图像坐标以及所述车辆三维模型，利用投影公式确定所述备选二维关键点在世界坐标系下的备选世界坐标信息；所述世界坐标系包括以所述目标车辆的运动平面为坐标面的坐标系；
    根据目标模型点与备选模型点之间的相对位置关系、所述估计偏航角信息以及所述备选世界坐标信息，确定所述估计位置信息；所述目标模型点为所述目标二维关键点在所述车辆三维模型中对应的模型点，所述备选模型点为所述备选二维关键点在所述车辆三维模型中对应的模型点。
  9. 根据权利要求8所述的方法，其特征在于，通过下述方式确定所述备选二维关键点：
    利用卷积神经网络对所述待处理图像进行二维关键点检测，得到各二维关键点的图像坐标和置信度；
    将取置信度最大的二维关键点作为所述备选二维关键点。
  10. 根据权利要求1至9中任一项所述的方法，其特征在于，所述验证所述估计位姿是否满足验证条件，包括：
    确定匹配模型点在所述估计位姿下的投影图像坐标，所述匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述待处理图像中检测出来的所述目标车辆的二维关键点对应的模型点；
    基于所述匹配模型点的投影图像坐标与所述待处理图像中检测出来的二维关键点的图像坐标，计算重投影误差值；
    判断所述重投影误差值是否小于第一误差阈值。
  11. 根据权利要求1至10中任一项所述的方法，其特征在于，在所述验证所述估计位姿是否满足验证条件之后，所述方法还包括：
    若所述估计位姿未满足所述验证条件，则基于二维关键点信息、所述目标车辆匹配的 车辆三维模型，利用投影公式确定所述实际位姿；所述二维关键点信息是对所述待处理图像中的所述目标车辆进行二维关键点检测得到的。
  12. 根据权利要求1至11中任一项所述的方法，其特征在于，所述历史偏航角信息是根据历史图像确定的，所述历史偏航角信息与所述历史图像一一对应，所述历史图像的拍摄时刻早于所述待处理图像的拍摄时刻，所述历史图像包括所述目标车辆的图像，所述历史图像和所述待处理图像由同一相机拍摄；对应于拍摄时刻最早的历史图像的首个历史偏航角信息基于以下步骤确定：
    对拍摄时刻最早的历史图像中的目标车辆进行二维关键点检测，得到历史二维关键点的图像坐标；
    根据所述历史二维关键点，确定历史匹配模型点，所述历史匹配模型点为与所述目标车辆匹配的车辆三维模型中与所述历史二维关键点对应的模型点；
    确定初始估计位姿，并将所述初始估计位姿作为当前估计位姿；
    利用最小二乘法更新当前估计位姿并优化所述历史匹配模型点在当前估计位姿下的投影图像坐标与所述历史二维关键点的图像坐标之间的重投影误差值；
    将所述重投影误差值小于第二误差阈值时对应的偏航角信息确定为所述首个历史偏航角信息；或者
    将当前估计位姿更新次数大于次数阈值时对应的偏航角信息确定为所述首个历史偏航角信息。
  13. 一种电子设备，其特征在于，包括处理器以及存储器，所述存储器存储有计算机可读取指令，当所述计算机可读取指令由所述处理器执行时，运行权利要求1至12中任一项所述的方法。
  14. 一种可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时运行权利要求1至12中任一项所述的方法。
  15. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序，所述计算机程序包括程序指令，其特征在于，当所述程序指令被计算机执行时，所述计算机能够执行权利要求1至12中任一项所述的方法。
PCT/CN2022/108678 2021-08-13 2022-07-28 位姿确定方法、电子设备及可读存储介质 WO2023016271A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110931973.7 2021-08-13
CN202110931973.7A CN113793297B (zh) 2021-08-13 2021-08-13 位姿确定方法、装置、电子设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2023016271A1 true WO2023016271A1 (zh) 2023-02-16

Family

ID=79181815

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/108678 WO2023016271A1 (zh) 2021-08-13 2022-07-28 位姿确定方法、电子设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN113793297B (zh)
WO (1) WO2023016271A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115861975A (zh) * 2023-02-28 2023-03-28 杭州枕石智能科技有限公司 障碍物车辆位姿估计方法及设备
CN116906277A (zh) * 2023-06-20 2023-10-20 北京图知天下科技有限责任公司 风机偏航变动的确定方法、装置、电子设备及存储介质
CN117095131A (zh) * 2023-10-16 2023-11-21 广州紫为云科技有限公司 物体运动关键点的三维重建方法、设备及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113793297B (zh) * 2021-08-13 2024-10-18 北京迈格威科技有限公司 位姿确定方法、装置、电子设备及可读存储介质
CN114445335B (zh) * 2021-12-22 2024-04-12 武汉易思达科技有限公司 基于双目机器视觉的车辆行驶状态监测方法
CN114088103B (zh) * 2022-01-19 2022-05-20 腾讯科技（深圳）有限公司 车辆定位信息的确定方法和装置
CN114648639B (zh) * 2022-05-19 2022-08-23 魔视智能科技(武汉)有限公司 一种目标车辆的检测方法、系统及装置
CN116597012B (zh) * 2023-06-12 2024-07-16 深圳海星智驾科技有限公司 车辆位姿的识别方法、装置、计算机设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015064735A (ja) * 2013-09-25 2015-04-09 日産自動車株式会社 自車位置推定装置及び自車位置推定方法
US20180137644A1 (en) * 2016-11-11 2018-05-17 Qualcomm Incorporated Methods and systems of performing object pose estimation
CN110823224A (zh) * 2019-10-18 2020-02-21 中国第一汽车股份有限公司 一种车辆定位方法及车辆
CN112050806A (zh) * 2019-06-06 2020-12-08 北京初速度科技有限公司 一种移动车辆的定位方法及装置
CN112230211A (zh) * 2020-10-15 2021-01-15 长城汽车股份有限公司 车辆定位方法、装置、存储介质及车辆
CN112967339A (zh) * 2020-12-28 2021-06-15 北京市商汤科技开发有限公司 一种车辆位姿的确定方法、车辆控制方法及装置、车辆
CN113793297A (zh) * 2021-08-13 2021-12-14 北京迈格威科技有限公司 位姿确定方法、装置、电子设备及可读存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10332334A (ja) * 1997-06-04 1998-12-18 Hitachi Ltd 画像処理による位置計測方法および装置
CN109816704B (zh) * 2019-01-28 2021-08-03 北京百度网讯科技有限公司 物体的三维信息获取方法和装置
CN110111388B (zh) * 2019-05-10 2021-03-23 北京航空航天大学 三维物体位姿参数估计方法及视觉设备
CN112810603B (zh) * 2019-10-31 2022-08-26 华为云计算技术有限公司 定位方法和相关产品
US11373332B2 (en) * 2020-01-06 2022-06-28 Qualcomm Incorporated Point-based object localization from images
CN111366154B (zh) * 2020-03-26 2022-05-17 三一建筑机器人(西安)研究院有限公司 一种航向角确定方法、装置及电子设备
CN112184818B (zh) * 2020-10-09 2022-06-10 重庆邮电大学 基于视觉的车辆定位方法及应用其方法的停车场管理系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015064735A (ja) * 2013-09-25 2015-04-09 日産自動車株式会社 自車位置推定装置及び自車位置推定方法
US20180137644A1 (en) * 2016-11-11 2018-05-17 Qualcomm Incorporated Methods and systems of performing object pose estimation
CN112050806A (zh) * 2019-06-06 2020-12-08 北京初速度科技有限公司 一种移动车辆的定位方法及装置
CN110823224A (zh) * 2019-10-18 2020-02-21 中国第一汽车股份有限公司 一种车辆定位方法及车辆
CN112230211A (zh) * 2020-10-15 2021-01-15 长城汽车股份有限公司 车辆定位方法、装置、存储介质及车辆
CN112967339A (zh) * 2020-12-28 2021-06-15 北京市商汤科技开发有限公司 一种车辆位姿的确定方法、车辆控制方法及装置、车辆
CN113793297A (zh) * 2021-08-13 2021-12-14 北京迈格威科技有限公司 位姿确定方法、装置、电子设备及可读存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115861975A (zh) * 2023-02-28 2023-03-28 杭州枕石智能科技有限公司 障碍物车辆位姿估计方法及设备
CN116906277A (zh) * 2023-06-20 2023-10-20 北京图知天下科技有限责任公司 风机偏航变动的确定方法、装置、电子设备及存储介质
CN117095131A (zh) * 2023-10-16 2023-11-21 广州紫为云科技有限公司 物体运动关键点的三维重建方法、设备及存储介质
CN117095131B (zh) * 2023-10-16 2024-02-06 广州紫为云科技有限公司 物体运动关键点的三维重建方法、设备及存储介质

Also Published As

Publication number Publication date
CN113793297B (zh) 2024-10-18
CN113793297A (zh) 2021-12-14

Similar Documents

Publication Publication Date Title
WO2023016271A1 (zh) 位姿确定方法、电子设备及可读存储介质
CN108027877B (zh) 用于非障碍区检测的系统和方法
KR102472767B1 (ko) 신뢰도에 기초하여 깊이 맵을 산출하는 방법 및 장치
CA3137709A1 (en) Image-based localization
EP3566172A1 (en) Systems and methods for lane-marker detection
JP7422105B2 (ja) 路側計算装置に用いる障害物３次元位置の取得方法、装置、電子デバイス、コンピュータ可読記憶媒体、及びコンピュータプログラム
CN111127524A (zh) 一种轨迹跟踪与三维重建方法、系统及装置
CN111260779B (zh) 地图构建方法、装置及系统、存储介质
WO2023016182A1 (zh) 位姿确定方法、装置、电子设备和可读存储介质
CN114898313B (zh) 驾驶场景的鸟瞰图生成方法、装置、设备及存储介质
CN111753739B (zh) 物体检测方法、装置、设备以及存储介质
CN113450579A (zh) 速度信息的获取方法、装置、设备及介质
CN111829522B (zh) 即时定位与地图构建方法、计算机设备以及装置
WO2023284358A1 (zh) 相机标定方法、装置、电子设备及存储介质
Jang et al. Camera orientation estimation using motion-based vanishing point detection for advanced driver-assistance systems
Dornaika et al. A new framework for stereo sensor pose through road segmentation and registration
CN116958195A (zh) 物件追踪整合方法及整合装置
KR20230049969A (ko) 글로벌 측위 장치 및 방법
CN113793250B (zh) 位姿评价方法、位姿确定方法、对应装置和电子设备
CN113902047B (zh) 图像元素匹配方法、装置、设备以及存储介质
WO2023076913A1 (en) Methods, storage media, and systems for generating a three-dimensional line segment
JP2023065296A (ja) 平面検出装置及び方法
CN113011212B (zh) 图像识别方法、装置及车辆
CN113763468A (zh) 一种定位方法、装置、系统及存储介质
CN112598736A (zh) 一种基于地图构建的视觉定位方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22855254

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 26.06.2024)