WO2021175119A1 - 用于获取车辆3d信息的方法和装置 - Google Patents
用于获取车辆3d信息的方法和装置 Download PDFInfo
- Publication number
- WO2021175119A1 WO2021175119A1 PCT/CN2021/076863 CN2021076863W WO2021175119A1 WO 2021175119 A1 WO2021175119 A1 WO 2021175119A1 CN 2021076863 W CN2021076863 W CN 2021076863W WO 2021175119 A1 WO2021175119 A1 WO 2021175119A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- vehicle
- projection
- width
- preset
- angle
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
- G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
- H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
- H04W4/46—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for vehicle-to-vehicle communication [V2V]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/12—Bounding box
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
Definitions
- This application relates to the field of automatic driving target detection, and more specifically, to a method and device for obtaining vehicle 3D information.
- 2D target detection With the application of neural networks in target detection tasks, two-dimensional (2D) target detection has reached a good detection level.
- ordinary 2D target detection cannot output all the information needed in actual application scenarios.
- 2D target detection can only output the corresponding 2D position and position of the target object in the picture. Confidence of belonging to different categories.
- objects In three-dimensional (3D) world perception, objects are three-dimensional, and most application scenarios require information such as the length, width, height, distance, orientation angle, and observation angle of the target object.
- the 3D target information Prediction has very important applications in the field of autonomous driving.
- the existing 3D target information prediction mainly relies on sensors such as monocular, binocular cameras and multi-line lidar.
- the prediction of 3D target information based on laser and the prediction of 3D target information based on monocular vision and neural network.
- the cost of predicting 3D target information based on laser is high and the operation is complicated, and the 3D based on monocular vision and neural network It is inconvenient to obtain and use the forecast data of the target information. Therefore, how to improve the accuracy of vehicle 3D information acquisition under the premise of controlling costs has become an urgent problem to be solved.
- the present application provides a method and device for obtaining vehicle 3D information, which are applied in the field of automatic driving target detection, which can improve the accuracy of vehicle 3D information acquisition under the premise of controlling costs, which becomes an urgent problem to be solved.
- a method for obtaining 3D information of a vehicle is provided.
- the method for obtaining 3D information of a vehicle may be executed by a terminal device, or may also be executed by a chip or circuit provided in the terminal device. There is no restriction on this.
- the aforementioned terminal device may be a mobile terminal device near the first vehicle.
- a second vehicle a terminal device held by a pedestrian, etc.
- the vicinity of the first vehicle may refer to the front, rear, or side of the first vehicle, etc., where the first vehicle can be detected.
- the method for obtaining 3D information of a vehicle includes:
- the body boundary of the first vehicle including the boundary between the body of the first vehicle and the front of the vehicle, or the boundary of the boundary between the body of the first vehicle and the rear of the vehicle; determine the vehicle body boundary based on the boundary of the vehicle The observation angle of the first vehicle and/or the heading angle of the first vehicle.
- the method for obtaining 3D information of a vehicle provided by the present application first obtains the body boundary of the first vehicle, and then determines the first vehicle's Observation angle; and/or, determining the heading angle of the first vehicle based on the relationship between the body boundary line of the first vehicle and the heading angle of the first vehicle. Since it is easier to obtain the body boundary of the first vehicle than to obtain the observation angle and heading angle of the first vehicle, the difficulty in obtaining the observation angle and/or heading angle is reduced, and the accuracy of vehicle 3D information acquisition can be improved. .
- the above-mentioned determination of the observation angle of the first vehicle based on the body boundary of the first vehicle may be determined by calculation, or may also be determined by looking up a table, and the specific determination method is not limited in the embodiment of the present application.
- determining the observation angle of the first vehicle by calculation includes: inputting the value of the body boundary of the first vehicle based on a calculation formula that is satisfied between the body boundary of the first vehicle and the observation angle of the first vehicle, and calculating The value of the observation angle of the first vehicle;
- determining the observation angle of the first vehicle by looking up a table includes: establishing a query form based on the correspondence between the body boundary line of the first vehicle and the observation angle of the first vehicle, and obtaining the body of the first vehicle For the value of the dividing line, the value of the observation angle of the first vehicle is queried from the query table.
- the above-mentioned determination of the heading angle of the first vehicle based on the body boundary line of the first vehicle can be determined by calculation, and can also be determined by looking up a table, which will not be repeated here.
- the determining the observation angle of the first vehicle based on the body boundary line includes: firstly based on the body boundary line and a preset 2D detection frame The width determines the body length of the first vehicle or the projection of the body width of the first vehicle in the direction parallel to the image plane; then based on the body length of the first vehicle or the body width of the first vehicle in the direction parallel to the image plane The projection on the upper surface, the width of the preset 2D detection frame, and the preset aspect ratio of the first vehicle determine the observation angle of the first vehicle.
- the above-mentioned determination of the observation angle of the first vehicle based on the body boundary line of the first vehicle may first determine the body length of the first vehicle and/or the body width of the first vehicle based on the body boundary line of the first vehicle. Projection in a direction parallel to the image plane, and then based on the projection of the length of the body of the first vehicle and/or the width of the body of the first vehicle in the direction parallel to the image plane, the width of the preset 2D detection frame And the preset aspect ratio of the first vehicle jointly determine the observation angle of the first vehicle.
- the width of the preset 2D detection frame can be understood as the width of the 2D detection frame is known
- the preset aspect ratio of the first vehicle can be understood as the aspect ratio of the first vehicle is known, that is, based on the acquisition
- the projection of the body length of the first vehicle and/or the body width of the first vehicle in the direction parallel to the image plane and the known parameter values determined by the body dividing line of the first vehicle can determine the observation angle of the first vehicle , Provides a specific method for determining the observation angle of the first vehicle based on the body boundary of the first vehicle.
- the determination of the body length of the first vehicle or the projection of the body width of the first vehicle in a direction parallel to the image plane based on the body boundary of the first vehicle involved in the embodiment of the present application includes: obtaining The position (for example, coordinates) of the body boundary line of the first vehicle on the top view of the image corresponding to the first vehicle is based on the known preset width of the 2D detection frame (for example, the two edges of the 2D detection frame are in the The position on the top view of the image corresponding to the first vehicle) and the positional relationship of the 2D detection frame between the body of the first vehicle and the image corresponding to the first vehicle determine that the length of the body of the first vehicle or the width of the body of the first vehicle is parallel to the image
- the projection in the direction of the plane will be described in detail in the embodiment part in conjunction with the drawings, and will not be repeated here.
- the projection of the body width of the first vehicle in a direction parallel to the image plane, the width of the preset 2D detection frame, and the first vehicle preset The length-to-width ratio and the observation angle of the first vehicle satisfy Where ⁇ is the width of the preset 2D detection frame, ⁇ is the preset aspect ratio of the first vehicle, and a'is the projection of the body width of the first vehicle in a direction parallel to the image plane , The ⁇ is the observation angle.
- the relationship between the projection of the body width of the first vehicle in a direction parallel to the image plane and the observation angle of the first vehicle can be expressed as Indicates that the independent variable in the relational expression is the projection of the body width of the first vehicle in a direction parallel to the image plane, the dependent variable is the observation angle of the first vehicle, other parameters are known values, and the first vehicle’s
- the projection of the body width in the direction parallel to the image plane is determined based on the body boundary of the first vehicle, which is equivalent to providing the form of a specific relationship between the body boundary of the first vehicle and the observation angle of the first vehicle .
- the projection of the length of the body of the first vehicle in a direction parallel to the image plane, the width of the preset 2D detection frame, and the preset of the first vehicle The length-to-width ratio and the observation angle of the first vehicle satisfy Where ⁇ is the width of the preset 2D detection frame, ⁇ is the preset aspect ratio of the first vehicle, and b is the projection of the body length of the first vehicle in a direction parallel to the image plane, The ⁇ is the observation angle.
- the relationship between the projection of the body length of the first vehicle in the direction parallel to the image plane and the observation angle of the first vehicle can be expressed by the relation Indicates that the independent variable in the relationship is the projection of the body length of the first vehicle in a direction parallel to the image plane, the dependent variable is the observation angle of the first vehicle, other parameters are known values, and the first vehicle’s
- the projection of the body length in the direction parallel to the image plane is determined based on the body boundary line of the first vehicle, which is equivalent to providing the form of a specific relationship between the body boundary line of the first vehicle and the observation angle of the first vehicle .
- the determining the heading angle of the first vehicle based on the body boundary line includes: firstly based on the body boundary line and a preset 2D detection frame The width determines the projection of the body length of the first vehicle in a direction parallel to the image plane, and determines that the body width of the first vehicle is parallel to the image based on the width of the body boundary and the preset 2D detection frame Projection in the direction of the plane; then based on the projection of the length of the body of the first vehicle in the direction parallel to the image plane, the projection of the width of the body of the first vehicle in the direction parallel to the image plane, the preset 2D detection frame The angle between the two edges and the optical center and the preset aspect ratio of the first vehicle determine the orientation angle of the first vehicle.
- the aforementioned determination of the heading angle of the first vehicle based on the body boundary line of the first vehicle may be to first determine the projection of the body length of the first vehicle in a direction parallel to the image plane based on the body boundary line of the first vehicle. And determine the projection of the body width of the first vehicle in the direction parallel to the image plane based on the body boundary line and the width of the preset 2D detection frame, and then determine the body length of the first vehicle based on the determined body length in parallel to the image plane.
- the projection in the direction of the image plane, the projection of the body width of the first vehicle in the direction parallel to the image plane, the angle between the two edges of the preset 2D detection frame and the optical center, and the preset of the first vehicle The aspect ratios together determine the heading angle of the first vehicle.
- the width of the preset 2D detection frame can be understood as the width of the 2D detection frame is known
- the preset aspect ratio of the first vehicle can be understood as the aspect ratio of the first vehicle is known
- the first vehicle The projection of the length of the body in the direction parallel to the image plane is known, and the angle between the two edges of the preset 2D detection frame and the optical center is also known, that is, based on the body score of the first vehicle.
- the projection of the length of the body of the first vehicle in the direction parallel to the image plane and the projection of the width of the body of the first vehicle in the direction parallel to the image plane and the known parameter values determined by the boundary can determine the orientation of the first vehicle Angle provides a specific way to determine the heading angle of the first vehicle based on the body boundary of the first vehicle.
- the projection of the body width of the first vehicle in a direction parallel to the image plane, and the body length of the first vehicle in a direction parallel to the image plane The angle between the two edges of the projection and the preset 2D detection frame and the direction of the optical center and the preset aspect ratio of the first vehicle and the orientation angle of the first vehicle meet the requirements
- a is the projection of the body length of the first vehicle in a direction parallel to the image plane
- b is the projection of the body width of the first vehicle in a direction parallel to the image plane
- ⁇ is the 2D detection
- ⁇ is the angle between the right edge of the 2D detection frame and the direction of the optical center
- the projection of the body width of the first vehicle in a direction parallel to the image plane, the projection of the body length of the first vehicle in a direction parallel to the image plane, and the second The relationship satisfied between the heading angles of a vehicle can be expressed as Indicates that the independent variable in the relationship is the projection of the body width of the first vehicle in a direction parallel to the image plane, and the projection of the body length of the first vehicle in a direction parallel to the image plane, and the dependent variable is the first
- the heading angle of a vehicle other parameters are known values, and the projection of the body length of the first vehicle in the direction parallel to the image plane and the projection of the body width of the first vehicle in the direction parallel to the image plane are based on
- the determination of the body boundary of the first vehicle is equivalent to providing a form of a specific relational expression satisfied between the body boundary of the first vehicle and the direction angle of the first vehicle.
- the method further includes: determining the 3D detection frame information corresponding to the first vehicle based on the heading angle of the first vehicle and the body boundary line.
- the method for obtaining 3D information of a vehicle may also determine the corresponding direction of the first vehicle based on the obtained body boundary line of the first vehicle and the heading angle of the first vehicle determined based on the body boundary line of the first vehicle.
- the 3D detection frame information uses the body boundary of the first vehicle to increase the constraint on the prediction of the 3D detection frame information, which can improve the accuracy of acquiring the 3D detection frame information corresponding to the first vehicle.
- a device for acquiring 3D information of a vehicle includes a processor for implementing the function of the terminal device in the method described in the first aspect.
- the apparatus for acquiring vehicle 3D information may further include a memory, which is coupled with the processor, and the processor is configured to implement the function of the terminal device in the method described in the first aspect.
- the memory is used to store program instructions and data.
- the memory is coupled with the processor, and the processor can call and execute the program instructions stored in the memory to implement the function of the terminal device in the method described in the first aspect.
- the device for acquiring 3D information of a vehicle may further include a communication interface, and the communication interface is used for the device for acquiring 3D information of a vehicle to communicate with other devices.
- the transceiver may be a communication interface or an input/output interface.
- the device for acquiring 3D information of a vehicle includes a processor and a communication interface, which are used to implement the function of the terminal device in the method described in the first aspect above, and specifically include:
- the processor uses the communication interface to communicate with the outside;
- the processor is used to run a computer program, so that the device implements any one of the methods described in the first aspect.
- the exterior may be an object other than the processor, or an object other than the device.
- the communication interface may be an input/output interface, interface circuit, output circuit, input circuit, Pins or related circuits, etc.
- the processor can also be embodied as a processing circuit or a logic circuit.
- a computer-readable storage medium on which a computer program is stored.
- the communication device realizes the first aspect and any one of the possible implementation manners of the first aspect method.
- a computer program product containing instructions when the instructions are executed by a computer, the communication device realizes the first aspect and the method in any possible implementation manner of the first aspect.
- a communication system including the apparatus for acquiring 3D information of a vehicle and an access network device as shown in the second aspect.
- FIG. 1 is a schematic diagram of the V2X system.
- Figure 2 is a schematic diagram of a vehicle body demarcation line.
- Figure 3 is a schematic diagram of an observation angle and an orientation angle.
- Fig. 4 is a schematic flowchart of a method for obtaining 3D information of a vehicle provided by the present application.
- Fig. 5 is a schematic diagram of the relationship between a vehicle body boundary line and an observation angle provided by an embodiment of the present application.
- Fig. 6 is a schematic diagram of a relationship between a body boundary line and an orientation angle provided by an embodiment of the present application.
- FIG. 7 is a schematic diagram of a device 700 for acquiring 3D information of a vehicle provided by the present application.
- FIG. 8 is a schematic structural diagram of a second vehicle 800 applicable to an embodiment of the present application.
- references described in this specification to "one embodiment” or “some embodiments”, etc. mean that one or more embodiments of the present application include a specific feature, structure, or characteristic described in combination with the embodiment. Therefore, the sentences “in one embodiment”, “in some embodiments”, “in some other embodiments”, “in some other embodiments”, etc. appearing in different places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless it is specifically emphasized otherwise.
- the terms “including”, “including”, “having” and their variations all mean “including but not limited to”, unless otherwise specifically emphasized.
- V2X vehicle to everything
- V2V vehicle to vehicle
- V2I vehicle to infrastructure
- V2P vehicle to pedestrian
- V2N vehicle to network
- FIG. 1 is a schematic diagram of the V2X system.
- the schematic diagram includes V2V communication, V2P communication, and V2I/N communication.
- the vehicles communicate via V2V.
- the vehicle can broadcast its own speed, driving direction, specific location, whether emergency brakes are stepped on, and other information to surrounding vehicles. By obtaining this type of information, drivers of surrounding vehicles can better perceive traffic conditions outside the line of sight, so as to Pre-judge the dangerous situation in advance and then make an avoidance; the vehicle and the roadside infrastructure communicate through V2I, and the roadside infrastructure can provide various types of service information and data network access for the vehicle. Among them, non-stop charging, in-car entertainment and other functions have greatly improved traffic intelligence.
- Roadside infrastructure for example, roadside unit (RSU) includes two types: one is a terminal equipment type RSU. Because RSUs are distributed on the roadside, the RSU of this terminal equipment type is in a non-mobile state, and there is no need to consider mobility; the other is the RSU of the network equipment type.
- the RSU of this network device type can provide timing synchronization and resource scheduling for vehicles communicating with the network device.
- Vehicles and people for example, vehicles and pedestrians, vehicles and cyclists, vehicles and drivers, or vehicles and passengers
- V2N and the aforementioned V2I can be collectively referred to as V2I/N.
- FIG. 1 is only an exemplary schematic diagram shown for introducing the V2X system, and does not constitute any limitation to the present application.
- the number of vehicles, the number of pedestrians, and the number of infrastructure can be multiple, not the number shown in FIG. 1.
- FIG. 1 illustrates the applicable scenarios of the embodiments of the present application.
- the following briefly introduces several basic concepts involved in the technical solutions of the present application.
- the task of object detection is an important task in the field of machine vision.
- the purpose of target detection is to find out the object of interest in the image or video, and realize the output of the detection target's position and the object classification of the detection target at the same time.
- the method of outputting the object category of the detection target and the minimum bounding frame of the detection target on the image is called 2D target detection; also for example, outputting the object category of the detection target and the length, width, height, and height of the detection target in three-dimensional space.
- Target detection with information such as rotation angle is called 3D target detection.
- the image involved in this application refers to the image of the target object detected by the terminal device.
- the image of the detection target (first vehicle) detected by the detection vehicle (second vehicle) exemplarily, the display image of the first vehicle on the central control screen of the second vehicle.
- Target detection which can also be called target extraction
- target extraction is a target location based on target geometric and statistical features. It combines target location and recognition. Its accuracy and real-time are an important capability of the entire system. Especially in complex scenes, when multiple targets need to be processed in real time, automatic target extraction and recognition are particularly important.
- the target detection technology in the prior art can be used, or the position of the detection target and the object classification of the detection target can be obtained based on the target detection technology in the future technology. Therefore, this application does not describe the specific implementation of the target detection technology. , Just refer to the description in the prior art.
- the detection frame is a rectangular frame, which can be determined by the x1 and y1 axis coordinates of the upper left corner of the rectangle and the x2 and y2 axis coordinates of the lower right corner of the rectangle.
- the bounding box is predicted by the neural network.
- the neural network predicts the position and size of a detection frame relative to the reference point, the type of object in the detection frame and the confidence of whether there is an object, and Confidence of the object category.
- 3D information plays an important role in the field of automatic driving target detection, for example, predicting the intention of other vehicles and evaluating Risk of collision, etc.
- 3D information can be predicted by a neural network, but neural network training requires 3D annotation data. Among them, the observation angle and the orientation angle are difficult to estimate by the human eye, and it is difficult to label the angle information.
- the 3D marking information of the vehicle is generally matched with a laser sensor for point cloud matching.
- the 3D positioning frame and the global orientation angle of the vehicle are generated in the simulation software according to its own calibration parameters, but the generated positioning frame and orientation angle cannot be used directly because
- the driving dynamics of the car such as bumps
- 3D information labeling with lidar is expensive, and data from both the laser and the image is required for labeling, which is inconvenient to use.
- the boundary between the body and the front of the vehicle or the boundary between the body and the rear of the vehicle is called the body boundary.
- Fig. 2 is a schematic diagram of a car body dividing line. Assuming that the line of sight of the observation camera is parallel light, Fig. 2(a) is an image of the first vehicle, and Fig. 2(b) is a top view of the image.
- the body boundary of the first vehicle is displayed as a line segment a1a2 in the image of the first vehicle, and the line segment a1a2 is displayed as points A, A, perpendicular to the x-axis of the image
- the coordinates of is (x0,0)
- the body of the first vehicle shown in (a) and (b) in FIG. 2 is on the right side of the 2D detection frame.
- Figure 2 (a) and (b) assume that the line of sight of the observation camera is parallel light. If the line of sight of the observation camera is not parallel light, the body of the first vehicle is on the left side of the 2D detection frame. .
- the projection of the body length of the first vehicle in the direction parallel to the image plane, and the projection of the body width of the first vehicle in the direction parallel to the image plane and the boundary line of the body of the first vehicle on the image x axis The abscissa value of the point is related, that is, the projection of the body length of the first vehicle in the direction parallel to the image plane can be determined based on the body boundary of the first vehicle, and the body width of the first vehicle in the direction parallel to the image plane Projection.
- Fig. 3 is a schematic diagram of the observation angle and the orientation angle.
- the second vehicle is the vehicle where the observation camera is located, and the first vehicle is the vehicle being observed.
- this application provides a method for obtaining vehicle 3D information, which can infer 3D information from 2D visual information, and optimizes the solution of obtaining 3D information from monocular vision.
- Fig. 4 is a schematic flowchart of a method for obtaining 3D information of a vehicle provided by the present application. At least some of the following steps are included.
- the execution subject for acquiring the body boundary line of the first vehicle may be a terminal device near the first vehicle.
- a second vehicle near the first vehicle a terminal device held by a pedestrian near the first vehicle, and the like.
- the vicinity of the first vehicle may refer to the front, rear, or side of the first vehicle, etc., where the first vehicle can be detected.
- the first vehicle is the vehicle in front of the second vehicle
- the second vehicle can detect information related to the first vehicle (such as position, speed, etc.), and control the speed and/or steering of the second vehicle based on the information related to the first vehicle , So as to avoid collision with the first vehicle.
- the detection of the first vehicle by the second vehicle may be implemented by the on-board device (for example, T-box) in the second vehicle.
- the on-board device for example, T-box
- the second vehicle detects the first vehicle. You can refer to Existing solutions for vehicle detection in the field of autonomous driving.
- the boundary line of the first vehicle may be manually marked to obtain the body boundary line of the first vehicle
- the body boundary of the first vehicle is labeled on the acquired image of the first vehicle. It should be understood that, relative to the labeling of angle information, the labeling of length information is more accurate and easier to implement.
- the machine learning model may be trained to return to the boundary line to obtain the body boundary line of the first vehicle;
- a machine learning model is established, and the machine learning model is trained to obtain the body boundary of the first vehicle.
- the specific modeling method is not limited in the embodiment of the present application.
- the body boundary of the first vehicle may be obtained based on an existing solution, or the first vehicle may be obtained based on a solution provided in future technologies.
- the dividing line of the vehicle body may be obtained based on an existing solution, or the first vehicle may be obtained based on a solution provided in future technologies.
- acquiring the body boundary line of the first vehicle involved in the embodiments of the present application can be understood as acquiring the abscissa value of the corresponding point of the body boundary line of the first vehicle on the x-axis of the image.
- the observation angle and/or the heading angle of the first vehicle may be determined based on the body boundary line. That is, the method flow shown in FIG. 4 also includes S420, determining the observation angle and/or heading angle of the first vehicle.
- determining the observation angle of the first vehicle includes: determining the projection of the body length of the first vehicle in a direction parallel to the image plane based on the body boundary line and the width of a preset 2D detection frame ; Determine the observation angle of the first vehicle based on the projection of the length of the body of the first vehicle in a direction parallel to the image plane, the width of the preset 2D detection frame, and the preset aspect ratio of the first vehicle .
- the plane parallel to the image referred to in this application can also be understood as a plane perpendicular to the direction of the optical center, and the plane parallel to the image is the plane on which the front view of the image of the first vehicle is located.
- FIG. 5 is a schematic diagram of a relationship between a vehicle body boundary line and an observation angle provided by an embodiment of the present application.
- the aspect ratio of the vehicle under test (for example, the first vehicle) is ⁇ .
- ⁇ is the observation angle
- c is the length of the body of the first vehicle
- c′ and are the width of the rear (or front) of the first vehicle.
- b can be considered as the projection of the body of the first vehicle perpendicular to the observation direction
- a' is the projection of the rear (or front) of the first vehicle in the direction perpendicular to the observation, and is proportional to The width of the body of the first vehicle and the rear (or front) of the first vehicle in the image
- A is the projection of the body boundary line perpendicular to the viewing direction (coordinates are (x0,0))
- B is the right side of the 2D detection frame
- C is the projection of the left edge of the 2D detection frame perpendicular to the observation direction (coordinates (x1,0))
- a′ can be expressed as x2- x0
- b can be expressed as x0-x1.
- the relationship between the projection of the body width of the first vehicle in the direction parallel to the image plane and the observation angle can be obtained, including:
- ⁇ in the above formula is known, and its physical meaning is the width of the 2D detection frame.
- the relationship between the projection of the body length of the first vehicle in the direction parallel to the image plane and the observation angle includes:
- ⁇ in the above formula is known, and its physical meaning is the width of the 2D detection frame.
- the embodiment of the present application does not limit how to determine the width of the 2D detection frame.
- the 2D detection frame can be determined by referring to the method of determining the 2D detection frame in the current 2D target detection process, and then the width of the 2D detection frame can be determined.
- the 2D detection frame is output by artificial labeling or the target detection neural network, and then the width of the 2D detection frame is determined based on the difference of the corner coordinates of the 2D detection frame.
- ⁇ can be understood as a correction value.
- the first vehicle may be determined first.
- the projection of the body width of the vehicle in the direction parallel to the image plane, and the projection of the body width of the first vehicle in the direction parallel to the image plane and the corresponding relationship determine the observation angle of the first vehicle.
- the specific expression form of the corresponding relationship between the projection of the body width of the first vehicle in a direction parallel to the image plane and the observation angle of the first vehicle may be a calculation formula, for example, the above formula
- the observation angle of the first vehicle can be determined by calculation.
- the projection of the body width of the first vehicle in the direction parallel to the image plane is first determined, and then based on the body width of the first vehicle in the direction parallel to the image plane.
- the calculation formula that satisfies the projection in the direction parallel to the image plane and the observation angle of the first vehicle enter the value of the projection of the body width of the first vehicle in the direction parallel to the image plane, and calculate the observation of the first vehicle The value of the angle; or,
- the corresponding relationship between the projection of the body width of the first vehicle in the direction parallel to the image plane and the observation angle of the first vehicle can be a table, and the observation of the first vehicle can be determined by looking up the table.
- Angle based on the correspondence between the projection of the body width of the first vehicle in the direction parallel to the image plane and the observation angle of the first vehicle, create a table, and obtain that the body width of the first vehicle is parallel to the image plane. For the value of the projection in the direction, query the value of the observation angle of the first vehicle from the table.
- the cost of calibrating the observation angle is relatively high, and this application mainly involves obtaining the observation angle based on the body boundary line. However, it is not limited that the body boundary line cannot be determined based on the observation angle. There is a correspondence between the body boundary of the first vehicle and the observation angle of the first vehicle, so the observation angle of the first vehicle may also be obtained first, and then the body boundary of the first vehicle is determined based on the observation angle of the first vehicle.
- ⁇ can be understood as a correction value.
- the first vehicle may be determined first.
- the projection of the body length of the vehicle in the direction parallel to the image plane, and the projection of the body length of the first vehicle in the direction parallel to the image plane and the corresponding relationship determine the observation angle of the first vehicle.
- the specific expression form of the correspondence between the projection of the body length of the first vehicle in the direction parallel to the image plane and the observation angle of the first vehicle may be a calculation formula, for example, the above formula
- the observation angle of the first vehicle can be determined by calculation.
- the projection of the body length of the first vehicle in the direction parallel to the image plane is first determined.
- the calculation formula that satisfies the projection in the direction parallel to the image plane and the observation angle of the first vehicle enter the value of the projection of the body length of the first vehicle in the direction parallel to the image plane, and calculate the observation of the first vehicle The value of the angle; or,
- the corresponding relationship between the projection of the length of the body of the first vehicle in the direction parallel to the image plane and the observation angle of the first vehicle can be in the form of a table, and the observation of the first vehicle can be determined by looking up the table.
- Angle based on the correspondence between the projection of the body length of the first vehicle in the direction parallel to the image plane and the observation angle of the first vehicle, create a table to obtain that the body length of the first vehicle is parallel to the image plane
- For the value of the projection in the direction query the value of the observation angle of the first vehicle from the table.
- the cost of calibrating the observation angle is relatively high, and this application mainly involves obtaining the observation angle based on the body boundary line. However, it is not limited that the body boundary line cannot be determined based on the observation angle. There is a correspondence between the body boundary of the first vehicle and the observation angle of the first vehicle, so the observation angle of the first vehicle may also be obtained first, and then the body boundary of the first vehicle is determined based on the observation angle of the first vehicle.
- the projection of the length of the body of the first vehicle in the direction parallel to the image plane can also be determined based on the observation angle.
- Determine the body boundary of the first vehicle based on the projection of the body length of the first vehicle in the direction parallel to the image plane, specifically between the projection of the body length of the first vehicle in the direction parallel to the image plane and the observation angle Meet the following formula:
- this application provides a method of correcting the observation angle for short distances.
- the approximate distance of the target is obtained through the calibration parameters and the approximate touch point of the object.
- the approximate distance of the target can be used to correct the projection of the vehicle body or the rear of the vehicle in the direction perpendicular to the observation direction.
- the calibration parameters include the geometric model parameters of the camera imaging.
- the calibration parameters can be used to determine the relationship between the three-dimensional geometric position of a point on the surface of the space object and its corresponding point in the image; the midpoint of the lower edge of the object's 2D detection frame can be seen Make the object approximate the touch point. Because the touch point is known to be the height of the ground, the position of the point in the image and the calibration parameters can be used to estimate the position of the object in the 3D space.
- the aspect ratio of the first vehicle mentioned above is preset, and another measure to control the error is to preset different aspect ratios for different models, such as cars and trucks.
- the above-mentioned observation angle only represents the observation direction of the second vehicle to the first vehicle, and the heading angle is the angle between the driving direction of the vehicle (the direction of the optical center of the camera) and the heading direction of the vehicle.
- the heading angle in the vehicle body coordinate system plays a more important role in the control of automatic driving. The following will describe how to determine the heading angle of the first vehicle based on the body boundary of the first vehicle.
- determining the heading angle of the first vehicle includes: first determining that the vehicle body width of the first vehicle is in a direction parallel to the image plane based on the body boundary line of the first vehicle and the width of a preset 2D detection frame The projection of the body of the first vehicle and the length of the preset 2D detection frame determine the projection of the body width of the first vehicle in a direction parallel to the image plane, and then based on the projection of the first vehicle The projection of the body width in a direction parallel to the image plane, the projection of the body length of the first vehicle in a direction parallel to the image plane, and the projection of the body length of the first vehicle in a direction parallel to the image plane The angle between the two edges of the preset 2D detection frame and the optical center and the preset aspect ratio of the first vehicle determine the orientation angle of the first vehicle.
- a AB be the actual projection of the body length of the first vehicle in a direction parallel to the image plane
- A is the projection of the body boundary line perpendicular to the observation direction (coordinates are (x0,0))
- E is the projection of the right edge of the 2D detection frame perpendicular to the observation direction (coordinates are (x2,0))
- B is 2D
- a can be expressed as x0-x1
- b1 can be expressed as x2-x0.
- angles ⁇ and ⁇ between the two edges of the 2D detection frame and the optical center can be calculated by the position of the 2D detection frame in the image and the calibration parameters:
- c x is the position of the optical center on the x-axis of the image
- f x is the focal length of the camera
- x left is the position of the left edge of the 2D detection frame on the x-axis of the image
- x right is the position of the right edge of the detection frame on the x-axis of the image.
- ⁇ can first determine the contour point between the left edge of the 2D detection frame and the optical center-the angle between the line between point #1 and the optical center and the direction of the optical center is ⁇ ; ⁇ can first determine the equal height point between the right edge of the 2D detection frame and the optical center-point #2, the angle between the line between point #2 and the optical center and the direction of the optical center is ⁇ , point #1, point #2 Co-planar with Guangxin.
- ⁇ and ⁇ which can be determined with reference to the position of the 2D detection frame in the image and the calibration parameters in the current 2D target detection.
- the angle corresponding to the side length a of ⁇ ABC is The angle corresponding to the side length c of ⁇ ABC is
- the angle corresponding to the side length b1 of ⁇ ADE is The angle corresponding to the side length c′ of ⁇ ADE is
- a is the projection of the body length of the first vehicle in a direction parallel to the image plane
- b1 is the body boundary line
- ⁇ is the angle between the left edge of the 2D detection frame and the optical center
- ⁇ is The angle between the right edge of the 2D detection frame and the optical center
- ⁇ can be understood as a correction value.
- the projection of the body width of the first vehicle in a direction parallel to the image plane the projection of the body length of the first vehicle in a direction parallel to the image plane, and the orientation angle of the first vehicle
- the projection of the body width of the first vehicle in the direction parallel to the image plane and the projection of the body length of the first vehicle in the direction parallel to the image plane can be determined first, and then The orientation angle of the first vehicle is determined based on the projection of the body width of the first vehicle in the direction parallel to the image plane and the projection of the body length of the first vehicle in the direction parallel to the image plane and the corresponding relationship.
- the correspondence between the projection of the body width of the first vehicle in the direction parallel to the image plane and the projection of the body length of the first vehicle in the direction parallel to the image plane and the orientation angle of the first vehicle can be a calculation formula, for example, the above formula Then the heading angle of the first vehicle can be determined by calculation. Based on the body boundary line of the first vehicle, the projection of the body width of the first vehicle in the direction parallel to the image plane and the length of the body of the first vehicle in the direction parallel to the image plane are first determined.
- the projection in the direction of the image plane is based on the projection of the body width of the first vehicle in the direction parallel to the image plane and the projection of the body length of the first vehicle in the direction parallel to the image plane and the orientation angle of the first vehicle Calculate the value of the projection of the body width of the first vehicle in the direction parallel to the image plane and the projection of the body length of the first vehicle in the direction parallel to the image plane.
- the value of the orientation angle or,
- the specific expression form of the correspondence between the projection of the body width of the first vehicle in a direction parallel to the image plane and the projection of the body length of the first vehicle in a direction parallel to the image plane and the orientation angle of the first vehicle It can be a table, then the heading angle of the first vehicle can be determined by looking up the table, based on the projection of the body width of the first vehicle in the direction parallel to the image plane and the length of the body of the first vehicle in the direction parallel to the image plane. The corresponding relationship between the projection on the first vehicle and the orientation angle of the first vehicle is established, and a table is established to obtain the projection of the body width of the first vehicle in the direction parallel to the image plane and the body length of the first vehicle in parallel to the image plane The value of the projection in the direction of, query the value of the heading angle of the first vehicle from the table.
- the cost of calibrating the orientation angle is relatively high, and this application mainly involves obtaining the orientation angle based on the body boundary line.
- the body boundary line cannot be determined based on the orientation angle.
- the heading angle of the first vehicle may also be obtained first, and then the body line of the first vehicle is determined based on the heading angle of the first vehicle. That is, only the heading angle of the first vehicle needs to be predicted to determine the body boundary of the first vehicle, which reduces the number of predictions.
- the 3D detection frame information corresponding to the first vehicle may be determined based on the heading angle of the first vehicle and the body boundary line.
- (x, y, 1) are the coordinates of the point on the 2D detection frame of the first vehicle
- (X, Y, Z, 1) are the coordinates on the 3D frame in the 3D camera coordinate system
- K, R , T is the calibration parameter
- K is the camera internal parameter matrix
- R and T are external camera parameters
- R and T are the rotation and translation of the camera coordinate system relative to the vehicle body coordinate system.
- the remaining five unknowns are t x , t y , t z , Y, Z.
- a total of five information can solve all the corresponding unknown 3D information .
- a more fitting 3D inspection frame can be obtained.
- the apparatus for obtaining vehicle 3D information in the foregoing method embodiments may perform some or all of the steps in the embodiments. These steps or operations are only examples. The embodiments of the present application may also include performing other operations or various operations. Deformed.
- the method for acquiring 3D information of a vehicle is described in detail above with reference to FIG. 4, and the device for acquiring 3D information of a vehicle provided by an embodiment of the present application is described in detail below with reference to FIG. 7 and FIG. 8.
- FIG. 7 is a schematic diagram of an apparatus 700 for acquiring 3D information of a vehicle provided by the present application.
- the apparatus 700 includes a processing unit 710 and an acquiring unit 720.
- the acquiring unit 720 is configured to acquire the body boundary line of the first vehicle
- the processing unit 710 is configured to determine the observation angle of the first vehicle and/or the heading angle of the first vehicle based on the body boundary line.
- the processing unit 710 determining the observation angle of the first vehicle based on the body boundary line includes:
- the processing unit 710 determines the body length of the first vehicle and/or the projection of the body width of the first vehicle in a direction parallel to the image plane based on the body boundary line and the width of the preset 2D detection frame; the processing unit 710 is determined based on the projection of the body length of the first vehicle and/or the body width of the first vehicle in a direction parallel to the image plane, the width of the preset 2D detection frame, and the preset aspect ratio of the first vehicle The observation angle of the first vehicle.
- the processing unit 710 determining the heading angle of the first vehicle based on the body boundary line includes:
- the processing unit 710 determines the projection of the body length of the first vehicle in a direction parallel to the image plane based on the body boundary line and the preset width of the 2D detection frame; the processing unit 710 determines the projection of the body length of the first vehicle in a direction parallel to the image plane based on the body boundary line and the preset width
- the width of the 2D detection frame determines the projection of the body width of the first vehicle in the direction parallel to the image plane; the processing unit 710 is based on the projection of the body width of the first vehicle in the direction parallel to the image plane, The projection of the body length of the first vehicle in a direction parallel to the image plane, the angle between the two edges of the preset 2D detection frame and the optical center, and the preset aspect ratio of the first vehicle preset the first The heading angle of the vehicle.
- the processing unit 710 is further configured to determine the 3D detection frame information corresponding to the first vehicle based on the heading angle of the first vehicle and the body boundary line.
- the device 700 completely corresponds to the second vehicle in the method embodiment, and the device 700 may be the second vehicle in the method embodiment, or a chip or functional module inside the second vehicle in the method embodiment.
- the corresponding units of the device 700 are used to execute the corresponding steps executed by the second vehicle in the method embodiment shown in FIG. 4.
- the processing unit 710 in the device 700 is used to execute the steps related to the processing corresponding to the second vehicle in the method embodiment. For example, step S420 of determining the observation angle and/or heading angle of the first vehicle in FIG. 4 is performed.
- the acquiring unit 720 in the device 700 executes the steps of acquiring the second vehicle in the method embodiment. For example, step S410 of obtaining the body boundary line of the first vehicle in FIG. 4 is performed.
- the apparatus 700 may further include a step of the sending unit executing the second vehicle sending.
- the acquiring unit 720 and the sending unit may constitute a transceiver unit, and have the functions of receiving and sending at the same time.
- the processing unit 710 may be at least one processor.
- the sending unit can be a transmitter or an interface circuit.
- the acquiring unit 720 may be a receiver or an interface circuit. The receiver and transmitter can be integrated to form a transceiver or interface circuit.
- the device 700 may further include a storage unit for storing data and/or signaling.
- the processing unit 710, the sending unit, and the obtaining unit 720 may interact or couple with the storage unit, for example, read or call data in the storage unit. And/or signaling, so that the method of the above-mentioned embodiment is executed.
- Each of the above units can exist independently, or can be fully or partially integrated.
- FIG. 8 is a schematic structural diagram of a second vehicle 800 applicable to an embodiment of the present application, which can be used to implement the function of the second vehicle in the above-mentioned method for acquiring 3D information of a vehicle.
- the second vehicle 800 includes a processor 810, a memory 820, and a transceiver 830.
- the memory 820 stores instructions or programs
- the processor 830 is configured to execute the instructions or programs stored in the memory 820.
- the transceiver 830 is configured to execute the operations performed by the obtaining unit 710 in the apparatus 700 shown in FIG. 7.
- An embodiment of the present application also provides a communication system, which includes the aforementioned second vehicle and access network equipment.
- An embodiment of the present application also provides a communication system, which includes the aforementioned second vehicle, first vehicle, and access network equipment.
- the present application also provides a computer-readable storage medium that stores instructions in the computer-readable storage medium.
- the computer executes the execution of the second vehicle in the method shown in FIG. 4 above.
- This application also provides a computer program product containing instructions.
- the computer program product runs on a computer, the computer executes the steps performed by the first vehicle in the method shown in FIG. 4.
- the application also provides a chip including a processor.
- the processor is used to read and run a computer program stored in the memory to execute the corresponding operation and/or process performed by the second vehicle in the method for obtaining vehicle 3D information provided by this application.
- the chip further includes a memory, the memory and the processor are connected to the memory through a circuit or a wire, and the processor is used to read and execute the computer program in the memory.
- the chip further includes a communication interface, and the processor is connected to the communication interface.
- the communication interface is used to receive processed data and/or information, and the processor obtains the data and/or information from the communication interface, and processes the data and/or information.
- the communication interface may be an input/output interface, interface circuit, output circuit, input circuit, pin or related circuit on the chip.
- the processor may also be embodied as a processing circuit or a logic circuit.
- the above-mentioned chip can also be replaced with a chip system, which will not be repeated here.
- the disclosed system, device, and method can be implemented in other ways.
- the device embodiments described above are only illustrative.
- the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
- the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual conditions to achieve the objectives of the solutions of the embodiments.
- the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
- the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
- the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
- the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .
- the term "and/or” in this application is only an association relationship that describes associated objects, which means that there can be three types of relationships, for example, A and/or B, which can mean that A alone exists, and both A and B exist. , There are three cases of B alone.
- the character "/" in this document generally means that the associated objects before and after are in an "or” relationship; the term “at least one” in this application can mean “one” and "two or more", for example, A At least one of, B and C can mean: A alone exists, B alone exists, C alone exists, A and B exist alone, A and C exist at the same time, C and B exist at the same time, A and B and C exist at the same time, this Seven situations.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Evolutionary Computation (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
一种用于获取车辆3D信息的方法和装置。该方法包括:首先确定第一车辆的车身分界线(S410),然后基于该车身分界线确定该第一车辆的观测角和/或朝向角(S420)。所述方法通过车辆的车身分界线确定出车辆的观测角和/或朝向角,可以提高车辆3D信息获取的准确性。该用于获取车辆3D信息的方法可以应用于智能驾驶、智能网联车和车联网等场景中。
Description
本申请要求于2020年03月06日提交中国专利局、申请号为202010152507.4、申请名称为“用于获取车辆3D信息的方法和装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及自动驾驶目标检测领域,并且更具体地,涉及一种用于获取车辆3D信息的方法和装置。
随着神经网络在目标检测任务上的应用,二维(two dimensional,2D)目标检测已经达到了很好的检测水平。但是在无人驾驶、机器人、增强现实等具体应用场景下,普通的2D目标检测并不能输出实际应用场景下所需要的所有信息,2D目标检测只能输出目标物体在图片中对应的2D位置和属于不同类别的置信度。在三维(three dimensional,3D)世界感知当中,物体都是有三维立体的,大部分应用场景都需要有目标物体的长、宽、高、距离、朝向角和观测角等信息。
自动驾驶场景下,需要从图像中获取目标物体三维大小及朝向角度等信息,这些信息投影在俯视图下对于后续自动驾驶场景中的路径规划和车辆控制具有至关重要的作用,因此3D目标信息的预测在自动驾驶领域有很重要的应用。
现有的3D目标信息的预测主要是依赖于单目、双目相机和多线激光雷达等传感器完成的。例如,基于激光进行3D目标信息的预测、基于单目视觉和神经网络进行3D目标信息的预测,其中,基于激光进行3D目标信息的预测成本高且操作复杂,基于单目视觉和神经网络进行3D目标信息的预测数据的获取和使用操作都不方便。因此,如何在控制成本的前提下提高车辆3D信息获取的准确性成为亟待解决的问题。
发明内容
本申请提供一种用于获取车辆3D信息的方法和装置,应用在自动驾驶目标检测领域中,可以在控制成本的前提下提高车辆3D信息获取的准确性成为亟待解决的问题。
第一方面,提供了一种用于获取车辆3D信息的方法,该用于获取车辆3D信息的方法可以由终端设备执行,或者,也可以由设置于终端设备中的芯片或电路执行,本申请对此不作限定。
上述的终端设备可以是第一车辆附近的移动终端设备。例如,第二车辆、行人持有的终端设备等。其中,第一车辆附近可以指的是第一车辆前方、后方、或侧方等能够检测到第一车辆的方位。
该用于获取车辆3D信息的方法包括:
获取第一车辆的车身分界线,该车身分界线包括第一车辆的车身和车头分界处的分界线,或者第一车辆的车身和车尾分界处的分界线;基于所述车身分界线确定所述第一车辆的观测角和/或所述第一车辆的朝向角。
本申请提供的用于获取车辆3D信息的方法,通过先获取第一车辆的车身分界线,再基于第一车辆的车身分界线和第一车辆的观测角之间的关系,确定第一车辆的观测角;和/或,基于第一车辆的车身分界线和第一车辆的朝向角之间的关系,确定第一车辆的朝向角。由于第一车辆的车身分界线的获取相对于获取第一车辆的观测角和朝向角来说更为容易,从而降低观测角和/或朝向角的获取难度,可以提高车辆3D信息获取的准确性。
需要说明书的是,本申请实施例中,由于第一车辆的车身分界线和第一车辆的朝向角之间存在对应关系、第一车辆的车身分界线和第一车辆的观测角之间也存在对应关系,所以也可以先获取第一车辆的朝向角和/或第一车辆的观测角,再基于第一车辆的朝向角确定第一车辆的车身分界线;或者,基于第一车辆的观测角确定第一车辆的车身分界线。即只需要预测第一车辆的朝向角和第一车辆的观测角中的一个量,就可以确定第一车辆的车身分界线,降低了预测量的个数。
另外,上述的基于第一车辆的车身分界线确定第一车辆的观测角可以通过计算的方式确定,还可以通过查表的方式确定,本申请实施例中对于具体的确定方式不做限定。
例如,通过计算的方式确定第一车辆的观测角包括:基于第一车辆的车身分界线和第一车辆的观测角之间满足的计算公式,输入第一车辆的车身分界线的值,计算得到第一车辆的观测角的值;
还例如,通过查表的方式确定第一车辆的观测角包括:基于第一车辆的车身分界线和第一车辆的观测角之间满足的对应关系,建立查询表格,获取到第一车辆的车身分界线的值,从查询表格中查询到第一车辆的观测角得值。
同理,上述的基于第一车辆的车身分界线确定第一车辆的朝向角可以通过计算的方式确定,还可以通过查表的方式确定,这里不再赘述。
结合第一方面,在第一方面的某些实现方式中,所述基于所述车身分界线确定所述第一车辆的观测角包括:首先基于所述车身分界线和预设的2D检测框的宽度确定该第一车辆的车身长度或第一车辆的车身宽度在平行于图像平面的方向上的投影;然后基于该第一车辆的车身长度或第一车辆的车身宽度在平行于图像平面的方向上的投影、该预设的2D检测框的宽度和所述第一车辆预设的长宽比确定所述第一车辆的观测角。
示例性地,上述的基于第一车辆的车身分界线确定第一车辆的观测角可以是先基于第一车辆的车身分界线确定得到该第一车辆的车身长度和/或第一车辆的车身宽度在平行于图像平面的方向上的投影,然后再基于该第一车辆的车身长度和/或第一车辆的车身宽度在平行于图像平面的方向上的投影、该预设的2D检测框的宽度以及第一车辆预设的长宽比共同确定第一车辆的观测角。其中,预设的2D检测框的宽度可以理解为2D检测框的宽度为已知的,第一车辆预设的长宽比可以理解为第一车辆的长宽比为已知的,即基于获取到第一车辆的车身分界线确定得到的第一车辆的车身长度和/或第一车辆的车身宽度在平行于图像平面的方向上的投影以及已知的参数值能够确定第一车辆的观测角,提供了基于第一车辆的车身分界线确定第一车辆的观测角的具体方式。
需要说明的是,本申请实施例中涉及到的基于第一车辆的车身分界线确定得到该第一 车辆的车身长度或第一车辆的车身宽度在平行于图像平面的方向上的投影包括:获取第一车辆的车身分界线在该第一车辆对应的图像的俯视图上的位置(例如,坐标),根据已知的预设的2D检测框的宽度(例如,2D检测框的两个边缘在该第一车辆对应的图像的俯视图上的位置)以及第一车辆的车身与第一车辆对应的图像的2D检测框位置关系确定出第一车辆的车身长度或第一车辆的车身宽度在平行于图像平面的方向上的投影,实施例部分将结合附图进行详细说明,这里不再赘述。
结合第一方面,在第一方面的某些实现方式中,所述第一车辆的车身宽度在平行于图像平面的方向上的投影、预设的2D检测框的宽度和所述第一车辆预设的长宽比与所述第一车辆的观测角之间满足
其中,ω为所述预设的2D检测框的宽度,μ为所述第一车辆预设的长宽比,a′为所述第一车辆的车身宽度在平行于图像平面的方向上的投影,所述θ为所述观测角。
本申请提供的用于获取车辆3D信息的方法,第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的观测角之间满足的关系可以用关系式
表示,该关系式中的自变量为第一车辆的车身宽度在平行于图像平面的方向上的投影,因变量为第一车辆的观测角,其他参数为已知的值,而且第一车辆的车身宽度在平行于图像平面的方向上的投影基于第一车辆的车身分界线确定的,相当于提供了第一车辆的车身分界线和第一车辆的观测角之间满足的具体关系式的形式。
结合第一方面,在第一方面的某些实现方式中,所述第一车辆的车身长度在平行于图像平面的方向上的投影、预设的2D检测框的宽度和所述第一车辆预设的长宽比与所述第一车辆的观测角之间满足
其中,ω为所述预设的2D检测框的宽度,μ为所述第一车辆预设的长宽比,b为所述第一车辆的车身长度在平行于图像平面的方向上的投影,所述θ为所述观测角。
本申请提供的用于获取车辆3D信息的方法,第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的观测角之间满足的关系可以用关系式
表示,该关系式中的自变量为第一车辆的车身长度在平行于图像平面的方向上的投影,因变量为第一车辆的观测角,其他参数为已知的值,而且第一车辆的车身长度在平行于图像平面的方向上的投影基于第一车辆的车身分界线确定的,相当于提供了第一车辆的车身分界线和第一车辆的观测角之间满足的具体关系式的形式。
结合第一方面,在第一方面的某些实现方式中,所述基于所述车身分界线确定所述第一车辆的朝向角包括:首先基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度在平行于图像平面的方向上的投影,以及基于所述车身分界线和预设的 2D检测框的宽度确定所述第一车辆的车身宽度在平行于图像平面的方向上的投影;然后基于第一车辆的车身长度在平行于图像平面的方向上的投影、第一车辆的车身宽度在平行于图像平面的方向上的投影、预设的2D检测框的两个边缘分别与光心的夹角和所述第一车辆预设的长宽比确定所述第一车辆的朝向角。
示例性地,上述的基于第一车辆的车身分界线确定第一车辆的朝向角可以是先基于第一车辆的车身分界线确定第一车辆的车身长度在平行于图像平面的方向上的投影,以及基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身宽度在平行于图像平面的方向上的投影,然后基于确定得到的第一车辆的车身长度在平行于图像平面的方向上的投影、第一车辆的车身宽度在平行于图像平面的方向上的投影、预设的2D检测框的两个边缘分别与光心的夹角和所述第一车辆预设的长宽比共同确定第一车辆的朝向角。其中,预设的2D检测框的宽度可以理解为2D检测框的宽度为已知的,第一车辆预设的长宽比可以理解为第一车辆的长宽比为已知的,第一车辆的车身长度在平行于图像平面的方向上的投影为已知的,预设的2D检测框的两个边缘分别与光心的夹角也是已知的,即基于获取到第一车辆的车身分界线确定得到的第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的车身宽度在平行于图像平面的方向上的投影以及已知的参数值能够确定第一车辆的朝向角,提供了基于第一车辆的车身分界线确定第一车辆的朝向角的具体方式。
结合第一方面,在第一方面的某些实现方式中,所述第一车辆的车身宽度在平行于图像平面的方向上的投影、所述第一车辆的车身长度在平行于图像平面的方向上的投影、预设的2D检测框的两个边缘分别与光心方向的夹角和所述第一车辆预设的长宽比与所述第一车辆的朝向角之间满足
其中,a为所述第一车辆的车身长度在平行于图像平面的方向上的投影,b为所述第一车辆的车身宽度在平行于图像平面的方向上的投影、α为所述2D检测框的左边缘与光心方向的夹角、β为所述2D检测框的右边缘与光心方向的夹角,
为所述第一车辆的朝向角。
本申请提供的用于获取车辆3D信息的方法,第一车辆的车身宽度在平行于图像平面的方向上的投影、所述第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的朝向角之间满足的关系可以用关系式
表示,该关系式中的自变量为第一车辆的车身宽度在平行于图像平面的方向上的投影、所述第一车辆的车身长度在平行于图像平面的方向上的投影,因变量为第一车辆的朝向角,其他参数为已知的值,而且第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的车身宽度在平行于图像平面的方向上的投影是基于第一车辆的车身分界线确定的,相当于提供了第一车辆的车身分界线和第一车辆的朝向角之间满足的具体关系式的形式。
结合第一方面,在第一方面的某些实现方式中,所述方法还包括:基于所述第一车辆的朝向角和所述车身分界线确定所述第一车辆对应的3D检测框信息。
本申请提供的用于获取车辆3D信息的方法,还可以基于获取的第一车辆的车身分界线和基于该第一车辆的车身分界线确定的第一车辆的朝向角,确定第一车辆对应的3D检 测框信息,利用第一车辆的车身分界线增加对3D检测框信息预测的约束,可以提高第一车辆对应的3D检测框信息获取的准确性。
第二方面,提供一种用于获取车辆3D信息的装置,该用于获取车辆3D信息的装置包括处理器,用于实现上述第一方面描述的方法中终端设备的功能。
可选地,该用于获取车辆3D信息的装置还可以包括存储器,该存储器与该处理器耦合,该处理器用于实现上述第一方面描述的方法中终端设备的功能。
在一种可能的实现中,该存储器用于存储程序指令和数据。该存储器与该处理器耦合,该处理器可以调用并执行该存储器中存储的程序指令,用于实现上述第一方面描述的方法中终端设备的功能。
可选地,该用于获取车辆3D信息的装置还可以包括通信接口,该通信接口用于该用于获取车辆3D信息的装置与其它设备进行通信。当该用于获取车辆3D信息的装置为终端设备时,该收发器可以是通信接口,或,输入/输出接口。
在一种可能的设计中,该用于获取车辆3D信息的装置包括:处理器和通信接口,用于实现上述第一方面描述的方法中终端设备的功能,具体地包括:
该处理器利用该通信接口与外部通信;
该处理器用于运行计算机程序,使得该装置实现上述第一方面描述的任一种方法。
可以理解,该外部可以是处理器以外的对象,或者是该装置以外的对象。
在另一种实现方式中,该用于获取车辆3D信息的装置为芯片或芯片系统时,该通信接口可以是是该芯片或芯片系统上输入/输出接口、接口电路、输出电路、输入电路、管脚或相关电路等。该处理器也可以体现为处理电路或逻辑电路。
第三方面,提供一种计算机可读存储介质,其上存储有计算机程序,该计算机程序被通信装置执行时,使得该通信装置实现第一方面以及第一方面的任一可能的实现方式中的方法。
第四方面,提供一种包含指令的计算机程序产品,该指令被计算机执行时使得通信装置实现第一方面以及第一方面的任一可能的实现方式中的方法。
第五方面,提供了一种通信系统,包括第二方面所示的用于获取车辆3D信息的装置和接入网设备。
图1是V2X系统的示意图。
图2是一种车身分界线的示意图。
图3是一种观测角和朝向角的示意图。
图4是本申请提供的一种用于获取车辆3D信息的方法示意性流程图。
图5是本申请实施例提供的一种车身分界线与观测角之间关系的示意图。
图6是本申请实施例提供的一种车身分界线与朝向角之间关系的示意图。
图7是本申请提供的用于获取车辆3D信息的装置700的示意图。
图8是适用于本申请实施例的第二车辆800的结构示意图。
下面将结合附图,对本申请中的技术方案进行描述。
以下实施例中所使用的术语只是为了描述特定实施例的目的,而并非旨在作为对本申请的限制。如在本申请的说明书和所附权利要求书中所使用的那样,单数表达形式“一个”、“一种”、“所述”、“上述”、“该”和“这一”旨在也包括。例如“一个或多个”这种表达形式,除非其上下文中明确地有相反指示。还应当理解,在本申请以下各实施例中,“至少一个”、“一个或多个”是指一个、两个或两个以上。
在本说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此,在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例,而是意味着“一个或多个但不是所有的实施例”,除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”,除非是以其他方式另外特别强调。
本申请实施例的技术方案可以应用于车联网(vehicle to everything,V2X)通信系统。随着社会的不断发展,汽车的普及程度也越来越高,驾驶出行在给人们的出行带来便利的同时,也给人类社会带来一定负面影响,车辆数量迅速增加引起了城市交通拥堵、交通事故频发、环境质量变差等一系列问题。从人身安全、交通出行效率、环境保护以及经济效应等多方面来看,都需要一套完善的智能交通系统(intelligent transportation system,ITS)。而当前,ITS也理所当然的成为了全球关注热点。
目前,车辆可以通过车辆与车辆(vehicle to vehicle,V2V)、车辆与路边基础设施(vehicle to infrastructure,V2I)、车辆与行人之间(vehicle to pedestrian,V2P)或者车辆与网络(vehicle to network,V2N)通信方式,及时获取路况信息或接收服务信息,这些通信方式可以统称为V2X通信(X代表任何事物)。
图1是V2X系统的示意图。该示意图包括V2V通信、V2P通信以及V2I/N通信。如图1所示,车辆之间通过V2V通信。车辆可以将自身的车速、行驶方向、具体位置、是否踩了紧急刹车等信息广播给周围车辆,周围车辆的驾驶员通过获取该类信息,可以更好的感知视距外的交通状况,从而对危险状况做出提前预判进而做出避让;车辆与路侧基础设施通过V2I通信,路边基础设施,可以为车辆提供各类服务信息和数据网络的接入。其中,不停车收费、车内娱乐等功能都极大的提高了交通智能化。路边基础设施,例如,路侧单元(road side unit,RSU)包括两种类型:一种是终端设备类型的RSU。由于RSU分布在路边,该终端设备类型的RSU处于非移动状态,不需要考虑移动性;另一种是网络设备类型的RSU。该网络设备类型的RSU可以给与网络设备通信的车辆提供定时同步及资源调度。车辆与人(例如,车辆与行人、车辆与骑自行车的人、车辆与司机或车辆与乘客)通过V2P通信;车辆与网络通过V2N通信,V2N可以与上述的V2I统称为V2I/N。
应理解,图1只是为了介绍V2X系统而示出的一种示例性的示意图,不对本申请构成任何限定。例如,车辆数量、行人数量以及基础设施的数量可以为多个,并不是图1中所示的数量。
图1介绍了本申请实施例能够应用的场景,为了便于对本申请技术方案的理解,下面简单介绍本申请技术方案中涉及的几个基本概念。
1、目标检测。
目标检测(object detection)的任务是机器视觉领域中的重要任务。目标检测的目的是找出图像或视频中感兴趣的物体,并同时实现输出检测目标的位置和检测目标的物体分类。例如,输出检测目标的物体类别和检测目标在图像上的最小包围框的方式称为2D目标检测;还例如,输出检测目标的物体类别及检测目标的在三维空间中的长、宽、高和旋转角等信息的目标检测称为3D目标检测。
需要说明的是,本申请中涉及的图像指的是终端设备检测到的目标的物体的影像。例如,检测车辆(第二车辆)检测到的检测目标(第一车辆)的影像,示例性地,第一车辆在第二车辆中控屏中的显示图像。
目标检测,也可以称为目标提取,是一种基于目标几何和统计特征的目标定位,它将目标的定位和识别合二为一,其准确性和实时性是整个系统的一项重要能力。尤其是在复杂场景中,需要对多个目标进行实时处理时,目标自动提取和识别就显得特别重要。
随着计算机技术的发展和计算机视觉原理的广泛应用,利用计算机图像处理技术对目标进行实时跟踪研究越来越热门,对目标进行动态实时跟踪定位在智能化交通系统、智能监控系统、军事目标检测及医学导航手术中手术器械定位等方面具有广泛的应用价值。
本申请实施例中可以沿用现有技术中的目标检测技术,也可以基于未来技术中的目标检测技术获取检测目标的位置和检测目标的物体分类,因此本申请不对目标检测技术的具体实现进行描述,参考现有技术中的描述即可。
2、2D检测框。
在2D目标检测过程中,通常使用检测框(bounding box)来描述检测目标的位置。检测框是一个矩形框,可以由矩形左上角的x 1和y1轴坐标与右下角的x2和y2轴坐标确定。
应理解,在2D目标检测过程中,bounding box是通过神经网络预测出来的,神经网络会预测一个检测框相对于参考点的位置和大小,检测框内物体类别和是否有物体的置信度,以及物体类别置信度。
3、3D检测框。
在相机坐标系下包络车辆的最小长方体。可视化为映射到2D图像上的平行四边体。
4、3D信息。
3D信息(例如,其他车辆的朝向角信息、本车辆对其他车辆的观测角信息、或其他车辆的位置信息等)在自动驾驶目标检测领域中有重要作用,例如,预测其他车辆的意图,评估碰撞风险等。3D信息可用神经网络预测,但神经网络训练需要3D标注数据,其中,观测角和朝向角人眼难以估计,很难标注角度信息。
目前车辆的3D标注信息一般用激光传感器进行点云匹配,根据自身的标定参数在模拟软件中生成车的3D定位框和全局的朝向角,但是生成的定位框和朝向角并不能直接使用,因为车的行驶动力学原因(如颠簸)需要后期人力手动与照片进行匹配生成最后车辆的3D信息。并且用激光雷达进行3D信息标注价格昂贵,标注时需要激光和图像两方的数据,使用不方便。
5、车身分界线。
车辆的2D检测框中,车辆的车身和车头分界处的分界线或者车辆的车身和车尾分界处的分界线称为车身分界线。
如图2所示,图2是一种车身分界线的示意图。假设观测相机的视线为平行光,图2(a)为第一车辆的图像,图2(b)为该图像的俯视图。从图2(a)和(b)中可以看出,第一车辆的车身分界线在第一车辆的图像中显示为线段a1a2,该线段a1a2在垂直于图像x轴的显示为点A,A的坐标为(x0,0)、2D检测框的宽度表示为ω=x2-x1,其中,x2为该2D检测框右边边缘在垂直于图像x轴的点的横坐标值,x1为该2D检测框左边边缘在垂直于图像x轴的点的横坐标值。
具体地,图2中(a)和(b)所示的第一车辆的车身在2D检测框的右边。第一车辆的车身长度在平行于图像平面的方向上的投影表示为a=x2-x0、第一车辆的车身宽度在平行于图像平面的方向上的投影表示为a=x0-x1;同理,若第一车辆的车身在2D检测框的左边。第一车辆的车身长度在平行于图像平面的方向上的投影表示为a=x0-x2、第一车辆的车身宽度在平行于图像平面的方向上的投影表示为a=x1-x0。
另外,需要说明的是图2中(a)和(b)所示的是假设观测相机的视线为平行光,若观测相机的视线不为平行光,第一车辆的车身在2D检测框的左边。第一车辆的车身长度在平行于图像平面的方向上的投影表示为a=x0-x2、第一车辆的车身宽度在平行于图像平面的方向上的投影表示为a=x1-x0。
由上述可知,第一车辆的车身长度在平行于图像平面的方向上的投影,以及第一车辆的车身宽度在平行于图像平面的方向上的投影与第一车辆的车身分界线在图像x轴的点的横坐标值相关,即可以基于第一车辆的车身分界线确定第一车辆的车身长度在平行于图像平面的方向上的投影,以及第一车辆的车身宽度在平行于图像平面的方向上的投影。
6、观测角。
观测相机与车身质心的连线方向和车头的朝向方向的夹角。
7、朝向角。
车头朝向方向与观测相机光心方向的夹角。如图3所示,图3是一种观测角和朝向角的示意图。从图3中可以看出,第二车辆为观测相机所在的车辆,第一车辆为被观测的车辆。
上述的3D目标检测中需要进行3D信息标注,但是直接在图像上标注角度和3D框很难标注准确,而借助激光点云标注成本高。另外,只从单目视觉入手很难得到准确的3D信息。为了解决现有技术中存在的缺陷,本申请提供一种用于获取车辆3D信息的方法,能够从2D视觉信息推测3D信息,优化了从单目视觉入手获取3D信息的方案。
下面结合附图介绍本申请提供的用于获取车辆3D信息的方法。
图4是本申请提供的一种用于获取车辆3D信息的方法示意性流程图。至少包括以下步骤中的部分步骤。
S410,获取第一车辆的车身分界线。
本申请实施例中获取第一车辆的车身分界线的执行主体可以是第一车辆附近的终端设备。例如,第一车辆附近的第二车辆、第一车辆附近的行人持有的终端设备等。其中,第一车辆附近可以指的是第一车辆前方、后方、或侧方等能够检测到第一车辆的方位。
考虑到本申请能够应用于V2V场景下,以检测第一车辆的执行主体为第二车辆为例进行说明。
例如,第一车辆为第二车辆前方的车辆,第二车辆可以检测到第一车辆相关的信息(比 如位置、速度等),基于第一车辆相关的信息控制第二车辆的速度和/或转向,从而避免和第一车辆发生碰撞。
示例性地,第二车辆检测第一车辆,可以通过第二车辆中的车载装置(例如,T-box)实现,本申请实施例中对于第二车辆如何检测第一车辆并不限定,可以参考现有的自动驾驶领域中车辆检测的方案。
作为一种可能的实现方式,本申请实施例中第二车辆检测第一车辆之后,可以通过人工标注分界线,获取第一车辆的车身分界线;
例如,通过人工标注的方式,在获取到的第一车辆的图像上标注出第一车辆的车身分界线。应理解,相对于角度信息的标注,长度信息的标注更为准确和易实现。
作为另一种可能的实现方式,本申请实施例中第二车辆检测第一车辆之后,可以通过训练机器学习模型回归分界线,获取第一车辆的车身分界线;
例如,建立机器学习模型,对该机器学习模型进行训练获取第一车辆的车身分界线。本申请实施例中对于具体的建模方式不做限定。
应理解,本申请实施例中对于如何获取该第一车辆的车身分界线并不限定,可以基于现有的方案获取第一车辆的车身分界线,也可以基于未来技术中提供的方案获取第一车辆的车身分界线。
还应理解,本申请实施例中涉及的获取第一车辆的车身分界线可以理解为获取到第一车辆的车身分界线在图像x轴的对应点的横坐标值。
进一步地,在获取了第一车辆的车身分界线之后,本申请实施例中可以基于该车身分界线确定该第一车辆的观测角和/或朝向角。即图4所示的方法流程还包括S420,确定第一车辆的观测角和/或朝向角。
一种可能的实现方式,确定第一车辆的观测角包括:基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度在平行于图像平面的方向上的投影;基于所述第一车辆的车身长度在平行于图像平面的方向上的投影、预设的2D检测框的宽度和所述第一车辆预设的长宽比确定所述第一车辆的观测角。下面结合图5详细说明如何确定第一车辆的观测角。
本申请中涉及的平行于图像平面也可以理解为垂直于光心方向的平面,平行于图像平面为第一车辆的图像的前视图所位于的平面。
如图5所示,图5是本申请实施例提供的一种车身分界线与观测角之间关系的示意图。
从图5中可以看出,车辆的车身分界线在检测框中的位置与观测角相关。为了简化问题,做如下假设:
一、假设观测车辆(例如,第二车辆)的视线为平行光;
二、假设被测车辆(例如,第一车辆)的长宽比为μ。
图5中,θ为观测角,c为第一车辆的车身长度,c'和为第一车辆的车尾(或,车头)宽度。由于平行光假设,可以认为b为第一车辆的车身为在垂直于观测方向的投影,a′为第一车辆的车尾(或,车头)为在垂直于观测方向的投影,并且等比于图像中第一车辆的车身的宽度和第一车辆的车尾(或,车头),A为车身分界线在垂直于观测方向的投影(坐标为(x0,0)),B为2D检测框右边边缘在垂直于观测方向的投影(坐标为(x2,0)),C为2D检测框左边边缘在垂直于观测方向的投影(坐标为(x1,0)),则a′可以表示为 x2-x0,b可以表示为x0-x1。
由图5可以得到三角形abc和三角形a'b'c'两个三角形为相似三角形,而第一车辆的长宽比μ即为这两个三角形的相似比,即:
从而可以得到第一车辆的车身宽度在平行于图像平面的方向上的投影和观测角度的关系包括:
其中,上式中的ω为已知的,物理意义为2D检测框的宽度。
对该公式进行简单变换可得:
和/或,
可以得到第一车辆的车身长度在平行于图像平面的方向上的投影和观测角度的关系包括:
其中,上式中的ω为已知的,物理意义为2D检测框的宽度。
对该公式进行简单变换可得:
本申请实施例中对于如何确定2D检测框的宽度并不限定,可以参考目前2D目标检测流程中确定2D检测框的方式确定出2D检测框,再确定出2D检测框的宽度。例如,通过人为标注或者由目标检测神经网络输出2D检测框,再基于2D检测框角点坐标做差值确定出2D检测框的宽度。
应理解,上述的关系式
中的a′=x2-x0,ω=x2-x1,则可以将该式变形为
其中,x0为第一车辆的车身分界线垂直于观测方向的投影,则该式可以理解为第一车辆的车身分界线和第一车辆的观测角之间存在的关系的一种具体形式,但是本申请中并不限定第一车辆的车身分界线和第一车辆的观测角之间存在的关系只能通过该关系式表示。
进一步地,由于第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的观测角之间存在对应关系,在获取到第一车辆的车身分界线之后,可以先确定第一车辆的车身宽度在平行于图像平面的方向上的投影,再基于第一车辆的车身宽度在平行于图像平面的方向上的投影和对应关系确定第一车辆的观测角。
具体地,上述第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的观测角之间存在的对应关系具体表现形式可以为计算公式,例如,上述的公式
则可以通过计算的方式确定第一车辆的观测角,基于第一车辆的车身分界线先确定第一车辆的车身宽度在平行于图像平面的方向上的投影,再基于第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的观测角之间满足的计算公式,输入第一车辆的车身宽度在平行于图像平面的方向上的投影的值,计算得到第一车辆的观测角的值;或者,
上述第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的观测角之间存在的对应关系具体表现形式可以为表格,则可以通过查表的方式确定第一车辆的观测角,基于第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的观测角之间满足的对应关系,建立表格,获取到第一车辆的车身宽度在平行于图像平面的方向上的投影的值,从表格中查询到第一车辆的观测角的值。
标定观测角成本较高,本申请中主要涉及基于车身分界线得到观测角度。但是,并不限定不可以基于观测角确定车身分界线。第一车辆的车身分界线和第一车辆的观测角之间存在对应关系,所以也可以先获取第一车辆的观测角,再基于第一车辆的观测角确定第一车辆的车身分界线。
由上述第一车辆的车身宽度在平行于图像平面的方向上的投影与观测角之间关系可知,还可以基于观测角确定第一车辆的车身宽度在平行于图像平面的方向上的投影,再基于该第一车辆的车身宽度在平行于图像平面的方向上的投影确定第一车辆的车身分界线,具体地第一车辆的车身宽度在平行于图像平面的方向上的投影与观测角之间满足如下公式:
第一车辆的车身宽度在平行于图像平面的方向上的投影与第一车辆的车身分界线之间满足如下公式:
a′=x2-x0,
即只需要预测第一车辆的观测角,就可以确定第一车辆的车身分界线,降低了预测量的个数。
同理,上述的关系式
中的b=x0-x1,ω=x2-x1,则可以将该式变形为
其中,x0为第一车辆的车身分界线垂直于观测方向的投影,则该式可以理解为第一车辆的车身分界线和第一车辆的观测角之间存在的关系的一种具体形式,但是本申请中并不限定第一车辆的车身分界线和第一车辆的观测角之间存在的关系只能通过该关系式表示。
进一步地,由于第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的观测角之间存在对应关系,在获取到第一车辆的车身分界线之后,可以先确定第一车辆的车身长度在平行于图像平面的方向上的投影,再基于第一车辆的车身长度在平行于图像平面的方向上的投影和对应关系确定第一车辆的观测角。
具体地,上述第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的观测角之间存在的对应关系具体表现形式可以为计算公式,例如,上述的公式
则可以通过计算的方式确定第一车辆的观测角,基于第一车辆的车身分界线先确定第一车辆的车身长度在平行于图像平面的方向上的投影,在基于第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的观测角之间满足的计算公式,输入第一车辆的车身长度在平行于图像平面的方向上的投影的值,计算得到第一车辆的观测角的值;或者,
上述第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的观测角之间存在的对应关系具体表现形式可以为表格,则可以通过查表的方式确定第一车辆的观测角,基于第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的观测角之间满足的对应关系,建立表格,获取到第一车辆的车身长度在平行于图像平面的方向上的投影的值,从表格中查询到第一车辆的观测角的值。
标定观测角成本较高,本申请中主要涉及基于车身分界线得到观测角度。但是,并不限定不可以基于观测角确定车身分界线。第一车辆的车身分界线和第一车辆的观测角之间存在对应关系,所以也可以先获取第一车辆的观测角,再基于第一车辆的观测角确定第一车辆的车身分界线。
由上述第一车辆的车身长度在平行于图像平面的方向上的投影与观测角之间关系可知,还可以基于观测角确定第一车辆的车身长度在平行于图像平面的方向上的投影,再基于该第一车辆的车身长度在平行于图像平面的方向上的投影确定第一车辆的车身分界线,具体地第一车辆的车身长度在平行于图像平面的方向上的投影与观测角之间满足如下公式:
第一车辆的车身长度在平行于图像平面的方向上的投影与第一车辆的车身分界线之间满足如下公式:
b=x0-x1,
即只需要预测第一车辆的观测角,就可以确定第一车辆的车身分界线,降低了预测量的个数。
由于平行光假设在目标在比较近的距离会不成立(例如,第一车辆和第二车辆之间的距离较近,小于5m),本申请中对于近距离的情况提供一种观测角修正的方法:通过标定参数和物体近似触地点得到目标大致距离,通过该目标大致距离可以修正车身或车尾在垂直于观测方向上的投影。其中,标定参数包括相机成像的几何模型参数,标定参数可以用于确定空间物体表面某点的三维几何位置与其在图像中对应点之间的相互关系;物体2D检测框下边缘的中点可看作物体近似触地点。因为已知触地点为地面高度,所以可以通过该点在图像中的位置和标定参数估算物体在3D空间的位置。
另外上述的第一车辆的长宽比为预设的,则另一个控制误差的措施是对于不同的车型,如轿车,卡车等预设不同的长宽比。
上述的观测角仅仅代表第二车辆对第一车辆的观测方向,而朝向角是车辆行驶方向(相机的光心方向)与车辆朝向方向的夹角。实际上车体坐标系下的朝向角,在自动驾驶 的控制上有更重要的作用。下面将介绍如何基于第一车辆的车身分界线确定第一车辆的朝向角。
一种可能的实现方式,确定第一车辆的朝向角包括:首先基于第一车辆的车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身宽度在平行于图像平面的方向上的投影,和基于第一车辆的车身分界线和预设的2D检测框的长度确定所述第一车辆的车身宽度在平行于图像平面的方向上的投影,然后基于所述第一车辆的车身宽度在平行于图像平面的方向上的投影、所述第一车辆的车身长度在平行于图像平面的方向上的投影、所述第一车辆的车身长度在平行于图像平面的方向上的投影、预设的2D检测框的两个边缘分别与光心的夹角和所述第一车辆预设的长宽比确定所述第一车辆的朝向角。
如图6所示,图6是本申请实施例提供的一种车身分界线与朝向角之间关系的示意图。从图6中可以看出,车辆的车身分界线在检测框中的位置与朝向角相关。图6中所示的
为朝向角,令c=AC为实际的第一车辆的车身长度,c′=AD为实际的第一车辆的车尾(或,车头)宽度。令a=AB为实际的第一车辆的车身长度在平行于图像平面的方向上的投影,b1=AE为第一车辆的车身宽度(车尾或车头)在平行于图像平面的方向上的投影,A为车身分界线在垂直于观测方向的投影(坐标为(x0,0)),E为2D检测框右边边缘在垂直于观测方向的投影(坐标为(x2,0)),B为2D检测框左边边缘在垂直于观测方向的投影(坐标为(x1,0)),则a可以表示为x0-x1,b1可以表示为x2-x0。
上述的2D检测框的两个边缘与光心的夹角α和β可以通过2D检测框在图像中的位置和标定参数计算出来:
其中,c
x为光心在图像x轴位置,f
x为相机的焦距,x
left为2D检测框的左边缘在图像x轴位置,x
right为检测框右边缘在图像x轴位置。
应理解,确定α可以先确定2D检测框的左边缘与光心的等高点-点#1点#1与光心之间的连线与光心方向的夹角即α;同理,确定β可以先确定2D检测框的右边缘与光心的等高点-点#2,点#2与光心之间的连线与光心方向的夹角即β,点#1、点#2和光心共面。本申请实施例中对于如何确定上述的α和β并不限定,可以参考目前2D目标检测中2D检测框在图像中的位置和标定参数确定。
对于图6中的三角形ΔABC应用正弦定理可得:
整理后可得:
同理,对于图6中的三角形ΔADE应用正弦定理可得:
整理后可得:
从而我们可以得到第一车辆的车身长度在平行于图像平面的方向上的投影、第一车辆的车身宽度在平行于图像平面的方向上的投影和朝向角度的关系为:
对该公式进行简单变换可得:
应理解,关系式
中的a=x0-x1,b1=x2-x0,则可以将该式变形为
其中,x0为第一车辆的车身分界线垂直于观测方向的投影,则该式可以理解为第一车辆的车身分界线和第一车辆的朝向角之间存在的关系的一种具体形式,但是本申请中并不限定第一车辆的车身分界线和第一车辆的朝向角之间存在的关系只能通过该关系式表示。
进一步地,由于第一车辆的车身宽度在平行于图像平面的方向上的投影、第一车辆的车身长度在平行于图像平面的方向上的投影和第一车辆的朝向角之间存在对应关系,在获取到第一车辆的车身分界线之后,可以先确定第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影,再基于第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影和对应关系确定第一车辆的朝向角。
具体地,上述第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车 身长度在平行于图像平面的方向上的投影和第一车辆的朝向角之间存在的对应关系具体表现形式可以为计算公式,例如,上述的公式
则可以通过计算的方式确定第一车辆的朝向角,基于第一车辆的车身分界线先确定第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影,再基于第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影与第一车辆的朝向角之间满足的计算公式,输入第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影的值,计算得到第一车辆的朝向角的值;或者,
上述第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影与第一车辆的朝向角之间存在的对应关系具体表现形式可以为表格,则可以通过查表的方式确定第一车辆的朝向角,基于第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影与第一车辆的朝向角之间满足的对应关系,建立表格,获取到第一车辆的车身宽度在平行于图像平面的方向上的投影和第一车辆的车身长度在平行于图像平面的方向上的投影的值,从表格中查询到第一车辆的朝向角的值。
标定朝向角成本较高,本申请中主要涉及基于车身分界线得到朝向角。但是,并不限定不可以基于朝向角确定车身分界线。第一车辆的车身分界线和第一车辆的朝向角之间存在对应关系,所以也可以先获取第一车辆的朝向角,再基于第一车辆的朝向角确定第一车辆的车身分界线。即只需要预测第一车辆的朝向角,就可以确定第一车辆的车身分界线,降低了预测量的个数。
进一步地,基于车身分界线确定上述的朝向角之后,可以基于基于所述第一车辆的朝向角和所述车身分界线确定所述第一车辆对应的3D检测框信息。
具体地,通过相机投影原理可以得到:
其中,(x,y,1)为第一车辆的2D检测框上的点2D检测框的坐标,(X,Y,Z,1)为3D相机坐标系下3D框上的坐标,K,R,T即为标定参数。K为相机内参矩阵。R,T为相机外参,R,T即相机坐标系相对于车体坐标系的旋转和平移。
假设滚动角度相对于水平面为零、倾斜角度相对于水平面为第一车辆长宽比,可以得到:
其中
为朝向角,剩下的五个未知数为t
x,t
y,t
z,Y,Z,基于2D检测框的四个边框 和车身分界线信息共五个信息可以解出对应的所有未知3D信息。通过车身分界线的约束,可以得到更贴合的3D检测框。
应理解,通过神经网络输出车辆的实际尺寸和观测角,从而推出观测目标的所有3D信息,会因为单目图片缺乏尺度信息,难以准确预测车辆在实际空间中的尺寸。与之不同的是,利用车身分界线来增加对3D信息预测的约束,并且对预测难度较高的车辆长宽,通过相机方程解出,而不是通过图片推测,提高确定3D信息的准确性。
应理解,上述方法实施例中用于获取车辆3D信息的装置可以执行施例中的部分或全部步骤,这些步骤或操作仅是示例,本申请实施例还可以包括执行其它操作或者各种操作的变形。
还应理解,在本申请的各个实施例中,如果没有特殊说明以及逻辑冲突,不同的实施例之间的术语和/或描述可以具有一致性、且可以相互引用,不同的实施例中的技术特征根据其内在的逻辑关系可以组合形成新的实施例。
上面结合图4详细介绍了用于获取车辆3D信息的方法,下面结合图7和图8详细介绍本申请实施例提供的用于获取车辆3D信息的装置。
参见图7,图7是本申请提供的用于获取车辆3D信息的装置700的示意图。如图7所示,装置700包括处理单元710、获取单元720。
获取单元720,用于获取第一车辆的车身分界线;
处理单元710,用于基于该车身分界线确定该第一车辆的观测角和/或该第一车辆的朝向角。
示例性地,该处理单元710基于该车身分界线确定该第一车辆的观测角包括:
该处理单元710基于该车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度和/或第一车辆的车身宽度在平行于图像平面的方向上的投影;该处理单元710基于所述第一车辆的车身长度和/或第一车辆的车身宽度在平行于图像平面的方向上的投影、预设的2D检测框的宽度和该第一车辆预设的长宽比确定该第一车辆的观测角。
示例性地,该处理单元710基于该车身分界线确定该第一车辆的朝向角包括:
该处理单元710基于该车身分界线和预设的2D检测框的宽度确定该第一车辆的车身长度在平行于图像平面的方向上的投影;该处理单元710基于该车身分界线和预设的2D检测框的宽度确定该第一车辆的车身宽度在平行于图像平面的方向上的投影;该处理单元710基于所述第一车辆的车身宽度在平行于图像平面的方向上的投影、所述第一车辆的车身长度在平行于图像平面的方向上的投影、预设的2D检测框的两个边缘分别与光心的夹角和该第一车辆预设的长宽比预设该第一车辆的朝向角。
示例性地,该处理单元710还用于基于该第一车辆的朝向角和该车身分界线确定该第一车辆对应的3D检测框信息。
装置700和方法实施例中的第二车辆完全对应,装置700可以是方法实施例中的第二车辆,或者方法实施例中的第二车辆内部的芯片或功能模块。装置700的相应单元用于执行图4所示的方法实施例中由第二车辆执行的相应步骤。
其中,装置700中的处理单元710用于执行方法实施例中第二车辆对应与处理相关的步骤。例如,执行图4中确定第一车辆的观测角和/或朝向角的步骤S420。
装置700中的获取单元720执行方法实施例中第二车辆获取的步骤。例如,执行图4 中获取第一车辆的车身分界线的步骤S410。装置700还可以包括发送单元执行第二车辆发送的步骤。
获取单元720和发送单元可以组成收发单元,同时具有接收和发送的功能。其中,处理单元710可以是至少一个处理器。发送单元可以是发射器或者接口电路。获取单元720可以是接收器或者接口电路。接收器和发射器可以集成在一起组成收发器或者接口电路。
可选的,装置700还可以包括存储单元,用于存储数据和/或信令,处理单元710、发送单元和获取单元720可以与存储单元交互或者耦合,例如读取或者调用存储单元中的数据和/或信令,以使得上述实施例的方法被执行。
以上各个单元可以独立存在,也可以全部或者部分集成。
参见图8,图8是适用于本申请实施例的第二车辆800的结构示意图,可以用于实现上述用于获取车辆3D信息的方法中的第二车辆的功能。该第二车辆800包括处理器810,存储器820与收发器830,其中,存储器820中存储指令或程序,处理器830用于执行存储器820中存储的指令或程序。存储器820中存储的指令或程序被执行时,收发器830用于执行图7所示的装置700中的获取单元710执行的操作。
本申请实施例还提供一种通信系统,其包括前述的第二车辆和接入网设备。
本申请实施例还提供一种通信系统,其包括前述的第二车辆、第一车辆和接入网设备。
本申请还提供了一种计算机可读存储介质,该计算机可读存储介质中存储有指令,当该指令在计算机上运行时,使得计算机执行上述如图4所示的方法中第二车辆执行的各个步骤。
本申请还提供了一种包含指令的计算机程序产品,当该计算机程序产品在计算机上运行时,使得计算机执行如图4所示的方法中第一车辆执行的各个步骤。
本申请还提供一种芯片,包括处理器。该处理器用于读取并运行存储器中存储的计算机程序,以执行本申请提供的用于获取车辆3D信息的方法中由第二车辆执行的相应操作和/或流程。可选地,该芯片还包括存储器,该存储器与该处理器通过电路或电线与存储器连接,处理器用于读取并执行该存储器中的计算机程序。进一步可选地,该芯片还包括通信接口,处理器与该通信接口连接。通信接口用于接收处理的数据和/或信息,处理器从该通信接口获取该数据和/或信息,并对该数据和/或信息进行处理。该通信接口可以是该芯片上的输入/输出接口、接口电路、输出电路、输入电路、管脚或相关电路等。所述处理器也可以体现为处理电路或逻辑电路。
上述的芯片也可以替换为芯片系统,这里不再赘述。
本申请中的术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(read-only memory,ROM)、随机存取存储器(random access memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
另外,本申请中术语“和/或”,仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,本文中字符“/”,一般表示前后关联对象是一种“或”的关系;本申请中术语“至少一个”,可以表示“一个”和“两个或两个以上”,例如,A、B和C中至少一个,可以表示:单独存在A,单独存在B,单独存在C、同时存在A和B,同时存在A和C,同时存在C和B,同时存在A和B和C,这七种情况。
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。
Claims (17)
- 一种用于获取车辆3D信息的方法,其特征在于,包括:获取第一车辆的车身分界线,其中,所述车身分界线包括所述第一车辆的车身和车头分界处的分界线,或者所述第一车辆的车身和车尾分界处的分界线;基于所述车身分界线确定所述第一车辆的观测角和/或所述第一车辆的朝向角。
- 如权利要求1所述的方法,其特征在于,所述基于所述车身分界线确定所述第一车辆的观测角包括:基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度或第一车辆的车身宽度在垂直于光心方向的平面上的投影;基于所述第一车辆的车身长度或第一车辆的车身宽度在垂直于光心方向的平面上的投影、所述预设的2D检测框的宽度和所述第一车辆预设的长宽比确定所述第一车辆的观测角。
- 如权利要求1-3中任一项所述的方法,其特征在于,所述基于所述车身分界线确定所述第一车辆的朝向角包括:基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度在平行于图像平面的方向上的投影;基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身宽度在平行于图像平面的方向上的投影;基于所述第一车辆的车身宽度在垂直于光心方向的平面上的投影、所述第一车辆的车身长度在垂直于光心方向的平面上的投影、所述预设的2D检测框的两个边缘分别与光心方向的夹角和所述第一车辆预设的长宽比确定所述第一车辆的朝向角。
- 如权利要求4或5所述的方法,其特征在于,所述方法还包括:基于所述第一车辆的朝向角和所述车身分界线确定所述第一车辆对应的3D检测框信息。
- 一种用于获取车辆3D信息的装置,其特征在于,包括:获取单元,用于获取第一车辆的车身分界线,所述车身分界线包括所述第一车辆的车身和车头分界处的分界线,或者所述第一车辆的车身和车尾分界处的分界线;处理单元,用于基于所述车身分界线确定所述第一车辆的观测角和/或所述第一车辆的朝向角。
- 如权利要求7所述的装置,其特征在于,所述处理单元基于所述车身分界线确定所述第一车辆的观测角包括:所述处理单元基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度或第一车辆的车身宽度在垂直于光心方向的平面上的投影;所述处理单元基于所述第一车辆的车身长度或第一车辆的车身宽度在垂直于光心方向的平面上的投影、所述预设的2D检测框的宽度和所述第一车辆预设的长宽比确定所述第一车辆的观测角。
- 如权利要求7-9中任一项所述的装置,其特征在于,所述处理单元基于所述车身分界线确定所述第一车辆的朝向角包括:所述处理单元基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身长度在平行于图像平面的方向上的投影;所述处理单元基于所述车身分界线和预设的2D检测框的宽度确定所述第一车辆的车身宽度在平行于图像平面的方向上的投影;所述处理单元基于所述第一车辆的车身宽度在垂直于光心方向的平面上的投影、所述第一车辆的车身长度在垂直于光心方向的平面上的投影、所述预设的2D检测框的两个边缘分别与光心方向的夹角和所述第一车辆预设的长宽比预设所述第一车辆的朝向角。
- 如权利要求10或11所述的装置,其特征在于,所述处理单元还用于基于所述第一车辆的朝向角和所述车身分界线确定所述第一车辆对应的3D检测框信息。
- 一种用于获取车辆3D信息的装置,其特征在于,包括:处理器和通信接口;其中,所述处理器通过所述接口电路与存储器耦合,所述处理器用于执行所述存储器中的程序代码,以实现如权利要求1-6中任一项所述的方法。
- 一种车载设备,其特征在于,用于实现如权利要求1-6中任意一项所述的方法。
- 一种通信系统,其特征在于,包括如权利要求7-12中任一项所述的装置,以及接入网设备。
- 一种芯片,包括通信接口、存储器和处理器,所述存储器用于存储计算机程序,所述处理器用于读取并执行所述存储器中存储器的所述计算机程序,使得所述芯片实现如权利要求1-6中任一项所述的方法。
- 一种计算机可读存储介质,其特征在于,包括:所述计算机可读介质存储有计算机程序;所述计算机程序在计算机上运行时,使得计算机执行权利要求1-6中任一项所述的方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP21765079.5A EP4105820A4 (en) | 2020-03-06 | 2021-02-19 | METHOD AND DEVICE FOR ACQUIRING 3D INFORMATION OF A VEHICLE |
US17/902,561 US20220414917A1 (en) | 2020-03-06 | 2022-09-02 | Method and apparatus for obtaining 3d information of vehicle |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010152507.4 | 2020-03-06 | ||
CN202010152507.4A CN113435224A (zh) | 2020-03-06 | 2020-03-06 | 用于获取车辆3d信息的方法和装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/902,561 Continuation US20220414917A1 (en) | 2020-03-06 | 2022-09-02 | Method and apparatus for obtaining 3d information of vehicle |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021175119A1 true WO2021175119A1 (zh) | 2021-09-10 |
Family
ID=77612983
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/076863 WO2021175119A1 (zh) | 2020-03-06 | 2021-02-19 | 用于获取车辆3d信息的方法和装置 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20220414917A1 (zh) |
EP (1) | EP4105820A4 (zh) |
CN (1) | CN113435224A (zh) |
WO (1) | WO2021175119A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113936458A (zh) * | 2021-10-12 | 2022-01-14 | 中国联合网络通信集团有限公司 | 高速公路拥堵判别方法、装置、设备及介质 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115876098B (zh) * | 2022-12-12 | 2023-10-24 | 苏州思卡信息系统有限公司 | 一种多线束激光雷达的车辆尺寸测量方法 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001128158A (ja) * | 1999-10-29 | 2001-05-11 | Ryosuke Shibazaki | 移動体用空間情報取得装置 |
CN102930242A (zh) * | 2012-09-12 | 2013-02-13 | 上海交通大学 | 一种公交车车型识别方法 |
CN109242903A (zh) * | 2018-09-07 | 2019-01-18 | 百度在线网络技术(北京)有限公司 | 三维数据的生成方法、装置、设备及存储介质 |
CN109631896A (zh) * | 2018-07-23 | 2019-04-16 | 同济大学 | 一种基于车辆视觉和运动信息的停车场自主泊车定位方法 |
CN110427797A (zh) * | 2019-05-28 | 2019-11-08 | 东南大学 | 一种基于几何条件限制的三维车辆检测方法 |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102019002269A1 (de) * | 2019-03-28 | 2020-02-27 | Daimler Ag | Verfahren zum Ermitteln einer Orientierung eines Fahrzeugs relativ zu einem Kraftfahrzeug |
CN110414355A (zh) * | 2019-06-27 | 2019-11-05 | 沈阳工业大学 | 基于视觉的右方位空停车位与泊车过程中车位线检测方法 |
CN110517349A (zh) * | 2019-07-26 | 2019-11-29 | 电子科技大学 | 一种基于单目视觉和几何约束的3d车辆目标检测方法 |
CN111081033B (zh) * | 2019-11-21 | 2021-06-01 | 北京百度网讯科技有限公司 | 车辆朝向角确定的方法及装置 |
-
2020
- 2020-03-06 CN CN202010152507.4A patent/CN113435224A/zh active Pending
-
2021
- 2021-02-19 WO PCT/CN2021/076863 patent/WO2021175119A1/zh unknown
- 2021-02-19 EP EP21765079.5A patent/EP4105820A4/en not_active Withdrawn
-
2022
- 2022-09-02 US US17/902,561 patent/US20220414917A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001128158A (ja) * | 1999-10-29 | 2001-05-11 | Ryosuke Shibazaki | 移動体用空間情報取得装置 |
CN102930242A (zh) * | 2012-09-12 | 2013-02-13 | 上海交通大学 | 一种公交车车型识别方法 |
CN109631896A (zh) * | 2018-07-23 | 2019-04-16 | 同济大学 | 一种基于车辆视觉和运动信息的停车场自主泊车定位方法 |
CN109242903A (zh) * | 2018-09-07 | 2019-01-18 | 百度在线网络技术(北京)有限公司 | 三维数据的生成方法、装置、设备及存储介质 |
CN110427797A (zh) * | 2019-05-28 | 2019-11-08 | 东南大学 | 一种基于几何条件限制的三维车辆检测方法 |
Non-Patent Citations (1)
Title |
---|
See also references of EP4105820A4 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113936458A (zh) * | 2021-10-12 | 2022-01-14 | 中国联合网络通信集团有限公司 | 高速公路拥堵判别方法、装置、设备及介质 |
Also Published As
Publication number | Publication date |
---|---|
EP4105820A4 (en) | 2023-07-26 |
CN113435224A (zh) | 2021-09-24 |
EP4105820A1 (en) | 2022-12-21 |
US20220414917A1 (en) | 2022-12-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11106893B1 (en) | System and method for evaluating the perception system of an autonomous vehicle | |
CN109461211B (zh) | 基于视觉点云的语义矢量地图构建方法、装置和电子设备 | |
EP3252658B1 (en) | Information processing apparatus and information processing method | |
CN110226186B (zh) | 表示地图元素的方法和装置以及定位的方法和装置 | |
CN110969064B (zh) | 一种基于单目视觉的图像检测方法、装置及存储设备 | |
WO2022104774A1 (zh) | 目标检测方法和装置 | |
CN108027877A (zh) | 用于非障碍区检测的系统和方法 | |
US11204610B2 (en) | Information processing apparatus, vehicle, and information processing method using correlation between attributes | |
WO2022161140A1 (zh) | 目标检测方法、装置、计算机设备及存储介质 | |
CN111595357B (zh) | 可视化界面的显示方法、装置、电子设备和存储介质 | |
CN114359181B (zh) | 一种基于图像和点云的智慧交通目标融合检测方法及系统 | |
WO2021175119A1 (zh) | 用于获取车辆3d信息的方法和装置 | |
CN114495064A (zh) | 一种基于单目深度估计的车辆周围障碍物预警方法 | |
CN111353453B (zh) | 用于车辆的障碍物检测方法和装置 | |
US11842440B2 (en) | Landmark location reconstruction in autonomous machine applications | |
EP3703008A1 (en) | Object detection and 3d box fitting | |
CN113869440A (zh) | 图像处理方法、装置、设备、介质及程序产品 | |
CN112639822B (zh) | 一种数据处理方法及装置 | |
CN116486351A (zh) | 行车预警方法、装置、设备及存储介质 | |
WO2021232222A1 (zh) | 用于测距的方法和装置 | |
WO2020100540A1 (ja) | 情報処理装置、情報処理システム、および情報処理方法、並びにプログラム | |
Fang et al. | Automatic roadblock identification algorithm for unmanned vehicles based on binocular vision | |
CN117470254B (zh) | 一种基于雷达服务的车载导航系统和方法 | |
US20220309693A1 (en) | Adversarial Approach to Usage of Lidar Supervision to Image Depth Estimation | |
CN114092916B (zh) | 图像处理方法、装置、电子设备、自动驾驶车辆及介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21765079 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2021765079 Country of ref document: EP Effective date: 20220916 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |