CN112017236B - Method and device for calculating target object position based on monocular camera - Google Patents
Method and device for calculating target object position based on monocular camera Download PDFInfo
- Publication number
- CN112017236B CN112017236B CN202010668524.3A CN202010668524A CN112017236B CN 112017236 B CN112017236 B CN 112017236B CN 202010668524 A CN202010668524 A CN 202010668524A CN 112017236 B CN112017236 B CN 112017236B
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- monocular camera
- vehicle
- point
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000004364 calculation method Methods 0.000 claims description 25
- 238000013519 translation Methods 0.000 claims description 25
- 230000015654 memory Effects 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 17
- 239000013598 vector Substances 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000003384 imaging method Methods 0.000 claims description 8
- 230000001131 transforming effect Effects 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000001174 ascending effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- G06T3/047—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention provides a method and a device for calculating the position of a target object based on a monocular camera, and relates to the technical field of road traffic; the method comprises the steps of calibrating internal and external parameters of a monocular camera in an off-line mode, acquiring a single-frame image of the monocular camera, positioning information of a vehicle in a map and local map information around the vehicle in real time, and calculating the real position of a target object based on the fact that the real position of the target object is fixed on a connecting ray of a center point of the monocular camera and a projection point of the target object in the same coordinate system. The method disclosed by the invention is completely independent of ground plane assumption, accurately calculates the real position of the target object, and is beneficial to the road running safety of the vehicle.
Description
Technical Field
The invention relates to the technical field of road traffic, in particular to a method and a device for calculating the position of a target object based on a monocular camera.
Background
The current target depth estimation method based on the monocular camera recovers the scale information of the monocular camera through the camera height calibrated in advance, and the method relies on the ground plane assumption that the target object and the plane of the vehicle are the same horizontal plane. The method has the problems that in an actual application scene, if a plane where a target object and a host vehicle are located does not meet the same level assumption, for example, a host vehicle jolt caused by ascending and descending slopes, a host vehicle passing through a deceleration strip or uneven road surfaces in the actual application scene and the plane where the target object is located are not located on the same level with the host vehicle, the projected position of the target object is estimated to be too close or too far, errors are related to the installation height and angle of a camera and the positions of the target object on other planes, the errors are not easy to model, and the erroneous position estimation seriously influences decision making in the downstream module planning control.
Disclosure of Invention
The invention aims to provide a method and a device for calculating the position of a target object based on a monocular camera, which can accurately calculate the real distance between the target object and a vehicle by detecting the grounding point of the target object and the ground plane where the target object is positioned and combining with the positioning map information of the vehicle, and solve the problems of over-distance and over-near estimated distance of the target object caused by the conditions that the target object is positioned on the same horizontal plane as the vehicle and the like due to the fact that the vehicle is bumpy due to ascending and descending slopes, over-deceleration strips or uneven pavement in an actual application scene without depending on ground plane assumption.
In order to achieve the above purpose, the present invention proposes the following technical scheme: a method of calculating a target position based on a monocular camera, the method comprising:
defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x-axis is rightward, and a z-axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
off-line calibrating inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system;
acquiring pixel positions of a target object and a ground point of a ground plane of the target object in the fish-eye original image, and marking the pixel positions as P tc The method comprises the steps of carrying out a first treatment on the surface of the According to the internal and external parameters of monocular camera, the monocular camera centre point P oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
Acquiring positioning information of a self vehicle, wherein the positioning information comprises the position of the self vehicle, and rotation and translation from a self vehicle body coordinate system to a map coordinate system; acquiring a vehicle pose according to the positioning information of the vehicle, and displaying a projection point P of a monocular camera center point in a vehicle body coordinate system of the vehicle ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
According to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit, acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object;
projection point P connecting center points of monocular cameras expressed in map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw Is denoted as ray L; and the intersection point of the ray L and the ground plane equation is the real position of the target object on the map coordinate system.
Further, the monocular camera center point P oc Pixel position P of target object and ground point of ground plane thereof under monocular camera coordinate system tc Transformed into a projection point P expressed under the coordinate system of the body of the self-vehicle ob Projection point P tb The calculation formula of (2) is as follows:
P tb ＝R c2b ·K -1 P tc +T c2b (1-1)
P ob ＝R c2b ·K -1 P oc +T c2b (1-2)
wherein ,Rc2b and Tc2b And respectively representing rotation and translation from the monocular camera coordinate system to the vehicle body coordinate system, wherein K is an internal parameter of off-line calibration of the monocular camera.
Further, the projection point P of the ground point of the ground plane where the target object and the ground plane where the target object are located are represented under the self-vehicle body coordinate system tb Projection point P of monocular camera center point ob Transformed into a map coordinate system expressed as a projection point P tw Projection point P ow The calculation formula of (2) is as follows:
P tw ＝R b2w ·P tb +T b2w (1-3)
P ow ＝R b2w ·P ob +T b2w (1-4)
wherein ,Rb2w and Tb2w Respectively representing rotation and translation from the vehicle body coordinate system to the map coordinate system.
Further, before the method is used for calibrating the internal and external parameters of the monocular camera offline, the method further comprises the following steps:
the monocular camera is connected to the rigid body of the vehicle, and the rotation and translation from the monocular camera coordinate system to the vehicle body coordinate system are fixed.
Further, a ground plane equation defining the ground plane where the target is located is:
A·x+b·y+C·z+D＝0 (1-5)
defining an origin of a map coordinate system as O, and defining an intersection point of a ray L and a ground plane equation of a target object as P; in the map coordinate system, the point P is over-projected ow Make a vertical line l to the ground plane of the target object 1 Vertical line l 1 The drop foot of the equation of the ground level where the target object is located is D, and the projection point P is exceeded tw To the vertical line l 1 Perpendicular line l 2 Vertical line l 2 From the perpendicular l 1 Drop foot is D 2 ；
The ground point of the object and the ground plane where the object is located is represented as:
wherein ,representing the vector between the original point O and the intersection point P in the map coordinate system +.>Representing the original point O and the projection point P in the map coordinate system ow Vector between->Representing projection point P in map coordinate system ow And projection point P tw Vectors between; />Respectively represent projection points P ow Coordinate components of an x axis, a y axis and a z axis in a map coordinate system;respectively represent projection points P tw Coordinate components of the x-axis, y-axis and z-axis in the map coordinate system.
The invention further discloses a device for calculating the position of a target object based on a monocular camera, which comprises: a processor for executing the following program modules stored in the memory;
the definition module is used for defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x axis is rightward, and a z axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
the calibration module is used for off-line calibrating the inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system; the internal parameters of the monocular camera are used for obtaining three-dimensional points of the target object recovered from the pixel position to the camera coordinate system;
a first obtaining module for obtaining the fish-eye original image at the ground point of the ground plane where the object in the fish-eye original image is locatedThe pixel position in (a) is denoted as P tc ；
A first transformation module for transforming the center point P of the monocular camera according to the internal and external parameters of the monocular camera oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
The second acquisition module is used for acquiring positioning information of the positioning unit on the own vehicle, and comprises the position of the own vehicle, and rotation and translation from a vehicle body coordinate system to a map coordinate system;
a second transformation module for acquiring the vehicle pose according to the positioning information of the vehicle and projecting a point P of the center point of the monocular camera expressed in the vehicle body coordinate system ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
The first calculation module is used for acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object according to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit;
a second calculation module for calculating a projection point P of the monocular camera center point expressed in the map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw Intersection points of the formed rays L and the ground plane equation; the intersection point is the real position of the target object on the map coordinate system.
Further, there is provided a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the method for calculating a target object position based on a monocular camera.
According to the technical scheme, the method for calculating the position of the target object based on the monocular camera provided by the technical scheme of the invention has the following beneficial effects:
according to the method and the device for calculating the position of the target object based on the monocular camera, the inner and outer parameters of the monocular camera are calibrated in an off-line mode, the single-frame image of the monocular camera, the positioning information of the vehicle in the map and the local map information around the vehicle are obtained in real time, and the real position of the target object is calculated based on the fact that the real position of the target object is fixed on a connecting ray of a center point of the monocular camera and a projection point of the target object in the same coordinate system. The method disclosed by the invention is completely independent of ground plane assumption, and can overcome the problems of excessive distance and excessive approaching estimated distance caused by situations that a vehicle passes through a deceleration strip or a road surface is uneven, the plane of a target object is not the same horizontal plane as the vehicle, and the like, which are caused by ascending and descending slopes appearing in actual application scenes; the invention can accurately calculate the real position of the target object, is beneficial to the self-vehicle to make an accurate decision in the navigation planning control of the downstream module, and ensures the road running safety of the self-vehicle.
It should be understood that all combinations of the foregoing concepts, as well as additional concepts described in more detail below, may be considered a part of the inventive subject matter of the present disclosure as long as such concepts are not mutually inconsistent.
The foregoing and other aspects, embodiments, and features of the present teachings will be more fully understood from the following description, taken together with the accompanying drawings. Other additional aspects of the invention, such as features and/or advantages of the exemplary embodiments, will be apparent from the description which follows, or may be learned by practice of the embodiments according to the teachings of the invention.
Drawings
The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of various aspects of the invention will now be described, by way of example, with reference to the accompanying drawings, in which:
FIG. 1 is a diagram showing the effect of the method of the present invention compared with the prior art;
FIG. 2 is a graph showing the calculation effect of the object of the present invention in a map coordinate system;
fig. 3 is a flow chart of the apparatus of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present invention fall within the protection scope of the present invention. Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs.
The terms "first," "second," and the like in the description and in the claims, are not used for any order, quantity, or importance, but are used for distinguishing between different elements. Also, unless the context clearly indicates otherwise, singular forms "a," "an," or "the" and similar terms do not denote a limitation of quantity, but rather denote the presence of at least one. The terms "comprises," "comprising," or the like are intended to cover a feature, integer, step, operation, element, and/or component recited as being present in the element or article that "comprises" or "comprising" does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. "up", "down", "left", "right" and the like are used only to indicate a relative positional relationship, and when the absolute position of the object to be described is changed, the relative positional relationship may be changed accordingly.
The method for judging the position of the target object by the target object depth estimation method of the monocular camera in the prior art depends on ground plane assumption and assumes the same plane where the target object and the vehicle are located, and when the target object and the vehicle are not in the same ground plane, the method can estimate the projected position of the target object too close or too far, so that the decision in planning control of a downstream module of the vehicle is seriously influenced. The invention provides a method for calculating the position of a target object based on a monocular camera, which can accurately calculate the real distance between the target object and a vehicle by detecting the grounding point of the target object and the ground plane where the target object is positioned and combining with the positioning map information of the vehicle, solves the problem of calculating the position of the target object when the plane where the target object is positioned is not the same horizontal plane as the vehicle, ensures the correct decision of a downstream module of the vehicle and improves the driving safety of a road.
The method for calculating the position of the target object based on the monocular camera according to the present invention will be described in detail with reference to the embodiments shown in the drawings.
Referring to fig. 1, in the prior art, a target depth estimation method using a monocular camera is adopted to estimate a point P at which a target is projected on a plane F, where the target and a host vehicle are assumed to be on the same plane F, based on that the real position of the target is fixed on a connecting ray between the center point of the monocular camera and the projection point of the target on the same coordinate system f Where the location belongs to the wrong location. In the actual vehicle running process, if the position of the target object is judged to be too far, the situation that the vehicle collides with the target object is likely to happen in the subsequent navigation decision of the vehicle, and the existence of the problem is obviously unfavorable for the driving safety of a road; therefore, the identification of the target position where the target and the own vehicle are not on the same ground plane is of great significance to safe driving on the road.
According to the method for calculating the position of the target object based on the monocular camera, the inner and outer parameters of the monocular camera are calibrated in an off-line mode, the single-frame image of the monocular camera, the positioning information of the vehicle in the map and the local map information around the vehicle are obtained in real time, and the real position of the target object is calculated based on the fact that the real position of the target object is fixed on a connecting ray of the center point of the monocular camera and the projection point of the target object in the same coordinate system.
The method specifically comprises the following steps:
defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x-axis is rightward, and a z-axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
off-line calibrating inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system;
acquiring pixel positions of a target object and a ground point of a ground plane of the target object in the fish-eye original image, and marking the pixel positions as P tc The method comprises the steps of carrying out a first treatment on the surface of the According to the internal and external parameters of monocular camera, the monocular camera centre point P oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
Acquiring positioning information of a self vehicle, wherein the positioning information comprises the position of the self vehicle, and rotation and translation from a self vehicle body coordinate system to a map coordinate system; acquiring a vehicle pose according to the positioning information of the vehicle, and displaying a projection point P of a monocular camera center point in a vehicle body coordinate system of the vehicle ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
According to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit, acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object;
projection point P connecting center points of monocular cameras expressed in map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw Is denoted as ray L; the intersection point of the ray L and the ground plane equation is the real position of the target object on the map coordinate system, i.e. the point P in fig. 1, and the ground plane where the target object is located in the method is the plane N in fig. 1.
In the invention, before the method is used for calibrating the internal and external parameters of the monocular camera offline, the method further comprises the following steps: the monocular camera is connected to the rigid body of the vehicle, and the rotation and translation from the monocular camera coordinate system to the vehicle body coordinate system are fixed.
In the method, the object and the grounding point of the ground plane where the object is positioned are firstly acquired by the monocular camera and recorded as pixel positions, and the pixel positions and the center point of the monocular camera are firstly transformed into a self-vehicle body coordinate system and then transformed into a map coordinate system through two-wheel transformation. The monocular camera center point P oc Pixel position P of target object and ground point of ground plane thereof under monocular camera coordinate system tc Transformed into a projection point P expressed under the coordinate system of the body of the self-vehicle ob Projection point P tb The calculation formula of (2) is as follows:
P tb ＝R c2b ·K -1 P tc +T c2b (1-1)
P ob ＝R c2b ·K -1 P oc +T c2b (1-2)
wherein ,Rc2b and Tc2b And respectively representing rotation and translation from the monocular camera coordinate system to the vehicle body coordinate system, wherein K is an internal parameter of off-line calibration of the monocular camera.
Projection point P of ground point of target object and ground plane where target object is located expressed under self-vehicle body coordinate system tb Projection point P of monocular camera center point ob Transformed into a map coordinate system expressed as a projection point P tw Projection point P ow The calculation formula of (2) is as follows:
P tw ＝R b2w ·P tb +T b2w (1-3)
P ow ＝R b2w ·P ob +T b2w (1-4)
wherein ,Rb2w and Tb2w Respectively representing rotation and translation from the vehicle body coordinate system to the map coordinate system.
Referring to fig. 2, after the image information and the positioning information are acquired, the invention starts to process the information internally, and the specific processing procedure is as follows: the ground plane equation defining the ground plane N where the target is located is:
A·x+b·y+C·z+D＝0 (1-5)
defining an origin of a map coordinate system as O, and defining an intersection point of a ray L and a ground plane equation of a target object as P; in the map coordinate system, the point P is over-projected ow Make a vertical line l to the ground plane of the target object 1 Vertical line l 1 The drop foot of the equation of the ground level where the target object is located is D, and the projection point P is exceeded tw To the vertical line l 1 Perpendicular line l 2 Vertical line l 2 From the perpendicular l 1 Drop foot is D 2 ；
The ground point of the object and the ground plane where the object is located is represented as:
wherein ,representing the vector between the original point O and the intersection point P in the map coordinate system +.>Representing the original point O and the projection point P in the map coordinate system ow Vector between->Representing projection point P in map coordinate system ow And projection point P tw Vectors between; />Respectively represent projection points P ow Coordinate components of an x axis, a y axis and a z axis in a map coordinate system;respectively represent projection points P tw Coordinate components of the x-axis, y-axis and z-axis in the map coordinate system. The above calculation process can obtain vector +.>The coordinates, O, are the origin of the map coordinate system, and the coordinates are (0, 0), so that the three-dimensional coordinates of the ground point of the target object and the ground plane where the target object is located in the map coordinate system can be directly obtained.
In addition, the method directly adopts the monocular camera as a sensing module, the single-frame image of the camera is input to the processor, the single-frame image of the camera extracts the pixel position information of the target object through methods such as image processing, deep learning and the like, the pixel position is usually the position of the target object and the grounding point of the ground plane where the target object is positioned in the image, and the ground plane does not require the assumption of the ground plane level of the vehicle. The own vehicle positioning information directly invokes sensor data and map data of a positioning unit and a map unit mounted on the own vehicle. The processor directly acquires various inertial sensors and image information which are input by the positioning unit and comprise the configuration of the vehicle, and calculates the position information and the attitude information of the vehicle in the map in real time. Because each vehicle is preconfigured with a loading map in the vehicle body system when running, and the local map around the vehicle and the target object is output by combining the positioning information of the vehicle input by the positioning unit, the ground information around the vehicle and the target object can be represented in a geometric equation parameterized form through a plurality of determined ground points describing the ground information in the local map, namely, the ground equation. A plane may be determined at three points that are not collinear, and therefore, when constructing a ground plane equation, the ground plane equation may be obtained by selecting three ground plane points that are not collinear from among the ground plane points determined by the local map. The existing map system can accurately present a local map of the position of the vehicle and the target object and determine the local map, and the map is reconstructed without re-acquiring data.
The method is completely independent of ground plane assumption, and can overcome the problems of over-long and over-short estimated distance of the target object caused by the situations that the vehicle runs on a slope, runs on a deceleration strip or the road surface is uneven, and the plane of the target object is not the same as the plane of the vehicle, and the like in the actual application scene; the invention can accurately calculate the real position of the target object, is beneficial to the self-vehicle to make an accurate decision in the navigation planning control of the downstream module, and ensures the road running safety of the self-vehicle.
An embodiment of the present invention provides a device for calculating a position of a target object based on a monocular camera, where the device obtains a real position of the target object by using the method for calculating a position of a target object based on a monocular camera disclosed above. For example, the method of calculating the position of the object based on the monocular camera may be divided into a plurality of modules, which are stored in a memory, and executed by a processor to accomplish the present invention. The plurality of modules or units may be a series of computer program instruction segments capable of performing a specific function for describing the execution of the method for calculating the position of the object based on the monocular camera in the device for calculating the position of the object based on the monocular camera. For example, the method for calculating the position of the target object based on the monocular camera can be divided into a definition module, a calibration module, a first acquisition module, a first transformation module, a second acquisition module, a second transformation module, a first calculation module and a second calculation module, and specific functions of each module are as follows:
the definition module is used for defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x axis is rightward, and a z axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
the calibration module is used for off-line calibrating the inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system; the internal parameters of the monocular camera are used to obtain a three-dimensional point of the object from the pixel position to the camera coordinate system.
A first obtaining module for obtaining the pixel position of the object and the ground point of the ground plane in the fish-eye original image, denoted as P tc ；
A first transformation module for transforming the center point P of the monocular camera according to the internal and external parameters of the monocular camera oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
The second acquisition module is used for acquiring positioning information of the positioning unit on the own vehicle, and comprises the position of the own vehicle, and rotation and translation from a vehicle body coordinate system to a map coordinate system;
a second transformation module for acquiring the vehicle pose according to the positioning information of the vehicle and projecting a point P of the center point of the monocular camera expressed in the vehicle body coordinate system ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
The first calculation module is used for acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object according to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit;
a second calculation module for calculating a projection point P of the monocular camera center point expressed in the map coordinate system ow Target and ground plane thereofProjection point P of grounding point of (2) tw Intersection points of the formed rays L and the ground plane equation; the intersection point is the real position of the target object on the map coordinate system.
Another embodiment of the present invention also provides an apparatus for calculating a position of a target object based on a monocular camera, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, i.e., the method for calculating a position of a target object based on a monocular camera. When the processor executes the computer program, the process of data acquisition and data calculation in the method for calculating the position of the target object based on the monocular camera is realized, as shown in fig. 2. In an embodiment, a computer program stored in a memory is divided into a plurality of modules, including a definition module, a calibration module, a first acquisition module, a first transformation module, a second acquisition module, a second transformation module, a first calculation module, and a second calculation module, and a processor executes the plurality of modules to complete the present invention. The specific functions of each module are as follows:
the definition module is used for defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x axis is rightward, and a z axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
the calibration module is used for off-line calibrating the inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system; the internal parameters of the monocular camera are used to obtain a three-dimensional point of the object from the pixel position to the camera coordinate system.
A first obtaining module for obtaining the pixel position of the object and the ground point of the ground plane in the fish-eye original image, denoted as P tc ；
First oneThe transformation module is used for transforming the center point P of the monocular camera according to the internal and external parameters of the monocular camera oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
The second acquisition module is used for acquiring positioning information of the positioning unit on the own vehicle, and comprises the position of the own vehicle, and rotation and translation from a vehicle body coordinate system to a map coordinate system;
a second transformation module for acquiring the vehicle pose according to the positioning information of the vehicle and projecting a point P of the center point of the monocular camera expressed in the vehicle body coordinate system ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
The first calculation module is used for acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object according to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit;
a second calculation module for calculating a projection point P of the monocular camera center point expressed in the map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw Intersection points of the formed rays L and the ground plane equation; the intersection point is the real position of the target object on the map coordinate system.
The device for calculating the target object position based on the monocular camera disclosed in the two embodiments may be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The means for calculating the position of the object based on the monocular camera may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that the apparatus schematic shown in fig. 2 is merely an example of an apparatus for calculating a position of a target object based on a monocular camera, and does not constitute a limitation of the apparatus for calculating a position of a target object based on a monocular camera, and may include more or less components than those shown, or may constitute some components, or different components, for example, the apparatus for calculating a position of a target object based on a monocular camera may further include an input/output device, a network access device, a bus, etc.
The processor may be a central processing unit of the host vehicle, but may also be other general purpose processors, digital signal trees, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the control center of the apparatus for calculating the position of the target object based on the monocular camera is connected with various interfaces and lines to the respective parts of the entire apparatus for calculating the position of the target object based on the monocular camera.
The memory is used as a non-transitory computer readable storage medium, and can be used for storing a non-transitory software program, a non-transitory computer executable program and a module, such as program instructions/modules corresponding to the method for calculating the position of the target object based on the monocular camera in the embodiment of the invention, and the processor executes various functional applications and data processing of the processor by running the non-transitory software program, instructions and modules stored in the memory, that is, the method for calculating the position of the target object based on the monocular camera in the embodiment of the method is realized.
The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. Furthermore, the memory is preferably, but not limited to, a high speed random access memory, for example, and may also be a non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may also optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The method for calculating the position of the target object based on the monocular camera disclosed by the invention can be stored in a computer readable storage medium when being realized as a software functional unit of a computer program and sold or used as an independent product. With this understanding, all or part of the flow of the method of the above embodiment of the present invention may also be implemented by a computer program, which may be stored in a computer readable storage medium, and which when executed by a processor, implements the steps and results of the above embodiment of the method. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; the storage medium may also comprise a combination of memories of the kind described above.
In addition, although the present invention does not specifically describe the pixel position P tc The calculation of the position after the secondary transformation to the monocular camera coordinate system is very common in the prior art, but the transformation calculation method of the image coordinate system to the known camera coordinate system, the vehicle body coordinate system and the world coordinate system is considered by the applicant to be unnecessary to be described in detail.
While the invention has been described with reference to preferred embodiments, it is not intended to be limiting. Those skilled in the art will appreciate that various modifications and adaptations can be made without departing from the spirit and scope of the present invention. Accordingly, the scope of the invention is defined by the appended claims.
Claims (8)
1. A method of calculating a position of a target object based on a monocular camera, the method comprising:
defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x-axis is rightward, and a z-axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
calibrating inner and outer parameters of monocular camera off-line, wherein, the inner parametersThe parameters include the focal length of the monocular camera and the monocular camera center point P oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system;
acquiring pixel positions of a target object and a ground point of a ground plane of the target object in the fish-eye original image, and marking the pixel positions as P tc The method comprises the steps of carrying out a first treatment on the surface of the According to the internal and external parameters of monocular camera, the monocular camera centre point P oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
Acquiring positioning information of a positioning unit on a self vehicle, wherein the positioning information comprises the position of the self vehicle, and rotation and translation from a self vehicle body coordinate system to a map coordinate system; acquiring a vehicle pose according to the positioning information of the vehicle, and displaying a projection point P of a monocular camera center point in a vehicle body coordinate system of the vehicle ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
According to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit, acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object;
projection point P of monocular camera center point expressed in map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw The constituted ray is denoted as ray L; and the intersection point of the ray L and the ground plane equation is the real position of the target object on the map coordinate system.
2. The method for calculating a target position based on a monocular camera according to claim 1, wherein the monocular camera center point P oc Pixel position P of target object and ground point of ground plane thereof under monocular camera coordinate system tc Transformed into a projection point P expressed under the coordinate system of the body of the self-vehicle ob Projection point P tb The calculation formula of (2) is as follows:
P tb ＝R c2b ·K -1 P tc +T c2b (1-1)
P ob ＝R c2b ·K -1 P oc +T c2b (1-2)
wherein ,Rc2b and Tc2b And respectively representing rotation and translation from the monocular camera coordinate system to the vehicle body coordinate system, wherein K is an internal parameter of off-line calibration of the monocular camera.
3. The method for calculating the position of a target object based on a monocular camera according to claim 1, wherein the projection point P of the ground point of the target object and the ground plane where the target object is located is represented in the vehicle body coordinate system tb Projection point P of monocular camera center point ob Transformed into a map coordinate system expressed as a projection point P tw Projection point P ow The calculation formula of (2) is as follows:
P tw ＝R b2w ·P tb +T b2w (1-3)
P t P ow ＝R b2w ·P ob +T b2w (1-4)
wherein ,Rb2w and Tb2w Respectively representing rotation and translation from the vehicle body coordinate system to the map coordinate system.
4. The method of calculating a target position based on a monocular camera of claim 1, further comprising, prior to calibrating the inner and outer parameters of the monocular camera off-line:
the monocular camera is connected to the rigid body of the vehicle, and the rotation and translation from the monocular camera coordinate system to the vehicle body coordinate system are fixed.
5. The method for calculating the position of a target object based on a monocular camera according to claim 1, wherein the ground plane equation defining the ground plane at which the target object is located is:
A·x+b·y+C·z+D＝0 (1-5)
defining the origin of a map coordinate system as O and rays LThe intersection point of the ground plane equation with the target object is P; in the map coordinate system, the point P is over-projected ow Make a vertical line l to the ground plane of the target object 1 Vertical line l 1 The drop foot of the equation of the ground level where the target object is located is D, and the projection point P is exceeded tw To the vertical line l 1 Perpendicular line l 2 Vertical line l 2 From the perpendicular l 1 Drop foot is D 2 ；
The ground point of the object and the ground plane where the object is located is represented as:
wherein ,representing the vector between the original point O and the intersection point P in the map coordinate system +.>Representing the original point O and the projection point P in the map coordinate system ow Vector between->Representing projection point P in map coordinate system ow And projection point P tw Vectors between;respectively represent projection points P ow Coordinate components of an x axis, a y axis and a z axis in a map coordinate system;respectively represent projection points P tw Coordinate components of the x-axis, y-axis and z-axis in the map coordinate system.
6. An apparatus for calculating a position of a target object based on a monocular camera, comprising:
the definition module is used for defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x axis is rightward, and a z axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
the calibration module is used for off-line calibrating the inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system; the internal parameters of the monocular camera are used for obtaining three-dimensional points of the target object recovered from the pixel position to the camera coordinate system;
a first obtaining module for obtaining the pixel position of the object and the ground point of the ground plane in the fish-eye original image, denoted as P tc ；
A first transformation module for transforming the center point P of the monocular camera according to the internal and external parameters of the monocular camera oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
The second acquisition module is used for acquiring positioning information of the positioning unit on the own vehicle, and comprises the position of the own vehicle, and rotation and translation from a vehicle body coordinate system to a map coordinate system;
a second transformation module for acquiring the vehicle pose according to the positioning information of the vehicle and projecting a point P of the center point of the monocular camera expressed in the vehicle body coordinate system ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
The first calculation module is used for acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object according to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit;
a second calculation module for calculating a projection point P of the monocular camera center point expressed in the map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw Intersection points of the formed rays L and the ground plane equation; the intersection point is the real position of the target object on the map coordinate system.
7. An apparatus for calculating a position of a target object based on a monocular camera, comprising: a processor for executing the following program modules stored in the memory;
the definition module is used for defining a monocular camera coordinate system, a vehicle body coordinate system and a map coordinate system; the monocular camera coordinate system takes a monocular camera center point as an origin, an x axis is rightward, and a z axis is vertical to a right-hand coordinate system of a monocular camera imaging plane; the self-vehicle body coordinate system takes the center of the self-vehicle as an origin, the x-axis is the front direction of the vehicle head, and the z-axis is the right-hand coordinate system in the direction right above the vehicle; the map coordinate system is a world coordinate system;
the calibration module is used for off-line calibrating the inner and outer parameters of the monocular camera, wherein the inner parameters comprise the focal length of the monocular camera and the center point P of the monocular camera oc Distortion parameters, including rotation and translation of the monocular camera coordinate system to the vehicle body coordinate system; internal reference of the monocular cameraThe number is used for obtaining a three-dimensional point of the target object from the pixel position to the camera coordinate system;
a first obtaining module for obtaining the pixel position of the object and the ground point of the ground plane in the fish-eye original image, denoted as P tc ；
A first transformation module for transforming the center point P of the monocular camera according to the internal and external parameters of the monocular camera oc Pixel position P tc Transformed to the coordinate system of the body of the self-vehicle to be expressed and respectively marked as projection points P ob Projection point P tb ；
The second acquisition module is used for acquiring positioning information of the positioning unit on the own vehicle, and comprises the position of the own vehicle, and rotation and translation from a vehicle body coordinate system to a map coordinate system;
a second transformation module for acquiring the vehicle pose according to the positioning information of the vehicle and projecting a point P of the center point of the monocular camera expressed in the vehicle body coordinate system ob Projection point P of ground point of object and ground plane where object is located tb Transformed to be represented under the map coordinate system and respectively marked as projection points P ow Projection point P tw ；
The first calculation module is used for acquiring a ground plane equation of a ground plane where a plurality of ground plane points in a local map of the periphery of the vehicle calculate a target object according to the positioning information provided by the positioning unit on the vehicle and the map information provided by the map unit;
a second calculation module for calculating a projection point P of the monocular camera center point expressed in the map coordinate system ow Projection point P of ground point of object and ground plane where object is located tw Intersection points of the formed rays L and the ground plane equation; the intersection point is the real position of the target object on the map coordinate system.
8. A computer readable storage medium storing a computer program, which, when executed by a processor, implements the method of calculating a target position based on a monocular camera according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010668524.3A CN112017236B (en) | 2020-07-13 | 2020-07-13 | Method and device for calculating target object position based on monocular camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010668524.3A CN112017236B (en) | 2020-07-13 | 2020-07-13 | Method and device for calculating target object position based on monocular camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112017236A CN112017236A (en) | 2020-12-01 |
| CN112017236B true CN112017236B (en) | 2023-10-31 |
Family
ID=73499734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010668524.3A Active CN112017236B (en) | 2020-07-13 | 2020-07-13 | Method and device for calculating target object position based on monocular camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112017236B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114581509A (en) * | 2020-12-02 | 2022-06-03 | 魔门塔(苏州)科技有限公司 | Target positioning method and device |
| EP4256522A4 (en) * | 2020-12-04 | 2024-02-14 | Hinge Health Inc | Object three-dimensional localizations in images or videos |
| EP4016456A1 (en) * | 2020-12-18 | 2022-06-22 | Augmented Robotics UG | Method and system for determining a three dimensional position |
| CN113592951A (en) * | 2021-07-14 | 2021-11-02 | 阿波罗智联(北京)科技有限公司 | Method and device for calibrating external parameters of vehicle-road cooperative middle-road side camera and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015024407A1 (en) * | 2013-08-19 | 2015-02-26 | 国家电网公司 | Power robot based binocular vision navigation system and method based on |
| CN108121941A (en) * | 2016-11-30 | 2018-06-05 | 上海联合道路交通安全科学研究中心 | A kind of object speed calculation method based on monitoring device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8831290B2 (en) * | 2012-08-01 | 2014-09-09 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for determining poses of vehicle-mounted cameras for in-road obstacle detection |
-
2020
- 2020-07-13 CN CN202010668524.3A patent/CN112017236B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015024407A1 (en) * | 2013-08-19 | 2015-02-26 | 国家电网公司 | Power robot based binocular vision navigation system and method based on |
| CN108121941A (en) * | 2016-11-30 | 2018-06-05 | 上海联合道路交通安全科学研究中心 | A kind of object speed calculation method based on monitoring device |
Non-Patent Citations (2)
| Title |
|---|
| 徐晓娟 ; 宋焕生 ; 赵倩倩 ; 徐昊 ; .基于单目序列图像的车辆三维信息的获取.电子设计工程.2015,(05),全文. * |
| 李华志 ; 江汉红 ; 张朝亮 ; .目标跟踪角度标定的单目法设计.电视技术.2007,(S1),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112017236A (en) | 2020-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112017236B (en) | Method and device for calculating target object position based on monocular camera | |
| US11024055B2 (en) | Vehicle, vehicle positioning system, and vehicle positioning method | |
| WO2020098316A1 (en) | Visual point cloud-based semantic vector map building method, device, and electronic apparatus | |
| CN113657224B (en) | Method, device and equipment for determining object state in vehicle-road coordination | |
| CN111046743B (en) | Barrier information labeling method and device, electronic equipment and storage medium | |
| US20230215187A1 (en) | Target detection method based on monocular image | |
| Parra et al. | Robust visual odometry for vehicle localization in urban environments | |
| TWI754808B (en) | Vehicle, vehicle positioning system, and vehicle positioning method | |
| CN113580134B (en) | Visual positioning method, device, robot, storage medium and program product | |
| WO2023065342A1 (en) | Vehicle, vehicle positioning method and apparatus, device, and computer-readable storage medium | |
| Petrovai et al. | A stereovision based approach for detecting and tracking lane and forward obstacles on mobile devices | |
| CN115410167A (en) | Target detection and semantic segmentation method, device, equipment and storage medium | |
| Manivannan et al. | Vision based intelligent vehicle steering control using single camera for automated highway system | |
| CN113450334B (en) | Overwater target detection method, electronic equipment and storage medium | |
| CN110197104B (en) | Distance measurement method and device based on vehicle | |
| CN111833443A (en) | Landmark position reconstruction in autonomous machine applications | |
| CN112902911B (en) | Ranging method, device, equipment and storage medium based on monocular camera | |
| CN114648639B (en) | Target vehicle detection method, system and device | |
| WO2023283929A1 (en) | Method and apparatus for calibrating external parameters of binocular camera | |
| US11514588B1 (en) | Object localization for mapping applications using geometric computer vision techniques | |
| WO2022133986A1 (en) | Accuracy estimation method and system | |
| Velat et al. | Vision based vehicle localization for autonomous navigation | |
| Tummala et al. | SmartDashCam: automatic live calibration for DashCams | |
| CN113834463A (en) | Intelligent vehicle side pedestrian/vehicle monocular depth distance measuring method based on absolute size | |
| US10726563B2 (en) | Visual odometry device for 3D vehicle motion estimation and visual odometry method for 3D vehicle motion estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |