CN114078246A

CN114078246A - Method and device for determining three-dimensional information of detection object

Info

Publication number: CN114078246A
Application number: CN202010803409.2A
Authority: CN
Inventors: 符张杰; 杨臻; 张维
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-08-11
Filing date: 2020-08-11
Publication date: 2022-02-22
Also published as: WO2022033089A1

Abstract

The application provides a method and a device for determining three-dimensional information of a detection object, relates to the field of intelligent driving, and is used for determining the three-dimensional information of the detection object. The method comprises the following steps: acquiring an image to be detected; the image to be detected comprises a first detection object; determining a passable area boundary of the first detection object and a touchdown line of the first detection object; the passable area boundary comprises a boundary of the first detection object in the image to be detected; the contact line is a connecting line of an intersection point of the first detection object and the ground; the three-dimensional information of the first detection object is determined according to the border of the passable area and the contact line, and the three-dimensional information of the detection object can be determined through the picture acquired by the monocular camera, so that the hardware cost for determining the three-dimensional information of the first detection object is reduced, and the calculated amount of the first vehicle in image processing is reduced.

Description

Method and device for determining three-dimensional information of detection object

Technical Field

The present application relates to the field of intelligent driving, and in particular, to a method and an apparatus for determining three-dimensional information of a detection object.

Background

In an intelligent driving scenario, a running vehicle (referred to as a first vehicle in this application) needs to detect the position of other surrounding vehicles (referred to as second vehicles in this application) relative to the first vehicle, the size of the second vehicle (the length of the vehicle, the width of the vehicle) and the direction of the second vehicle (the direction of the vehicle, the running direction, etc.) in real time, so that the first vehicle plans its own running route according to the information, and can predict and avoid dangerous behaviors that may be brought by other vehicles.

When a first vehicle determines the size, orientation, and position of a second vehicle relative to the first vehicle, three-dimensional (3D) information of the second vehicle needs to be determined. Currently, a first vehicle can determine 3D information of a second vehicle in a binocular 3D mode. However, the binocular 3D method requires a binocular camera, which is expensive; and the current algorithm used in the determination process of the 3D information of the second vehicle requires high image labeling cost and calculation amount.

Disclosure of Invention

The application provides a method and a device for determining three-dimensional information of a detection object, and solves the problems that in the prior art, when the three-dimensional information of surrounding vehicles is determined, the requirement on hardware is high, and the calculation data volume is large.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, a method for determining three-dimensional information of a detection object is provided, including: a first vehicle acquires an image to be detected; the image to be detected comprises a first detection object; the first vehicle determines the border of the passable area of the first detection object and the touchdown line of the first detection object; the passable area boundary comprises a boundary of the first detection object in the image to be detected; the contact line is a connecting line of an intersection point of the first detection object and the ground; the first vehicle determines the three-dimensional information of the first detection object according to the border of the passable area and the grounding line.

Based on the technical scheme, the method for determining the three-dimensional information of the detection object provided by the application includes that the first vehicle can determine the passable area boundary and the grounding line of the first detection object according to the collected image to be detected, and further, the first vehicle determines the three-dimensional information represented by the first detection object in the two-dimensional image according to the passable area boundary and the grounding line of the first detection object.

The image to be detected can be a two-dimensional image collected by a monocular camera. Therefore, the first vehicle can determine the three-dimensional information of the first detection object by acquiring the image information of the first detection object through the monocular camera. Compared with the method that the first vehicle needs to rely on the binocular camera to acquire the image information of the first detection object to determine the three-dimensional information of the first detection object in the prior art, the method that the monocular camera is adopted by the first vehicle to acquire the image information of the first detection object to determine the three-dimensional information of the first detection object in the application can greatly reduce the hardware cost for determining the three-dimensional information of the first detection object.

In addition, according to the method for determining the three-dimensional information of the detection object, the first vehicle only needs to mark the passable area boundary of the first detection object, and the contact line of the first detection object is determined according to the passable area boundary of the first detection object. The first vehicle can determine the three-dimensional information represented by the first detection object in the two-dimensional image according to the passable area boundary and the contact line of the first detection object and by combining the visual relationship and the like of the first detection object in the image to be detected. Therefore, the method for determining the three-dimensional information of the detection object does not need the first vehicle to perform other additional data annotation training, so that the calculation amount for determining the three-dimensional information of the first detection object is reduced, and Graphics Processing Unit (GPU) resources occupied by determining the three-dimensional information of the first detection object are reduced.

With reference to the first aspect, in one possible implementation manner, the three-dimensional information of the first detection object is used to determine at least one of a size, a direction, and a relative position of the first detection object.

For example, the first vehicle may convert three-dimensional information represented by the first detection object in the image to be detected into a body coordinate system of the first vehicle, and determine a corresponding size, a corresponding direction, and a corresponding position of the first detection object in the body coordinate system of the first vehicle. Therefore, the first vehicle can plan the driving route of the first vehicle by combining the corresponding size and direction of a plurality of first detection objects around in the vehicle body coordinate system of the first vehicle and the position of the first detection objects relative to the first vehicle, so as to realize intelligent driving of the first vehicle.

With reference to the first aspect, in a possible implementation manner, the passable area boundary of the first detection object includes a plurality of boundary points corresponding to the first identifier, and boundary points corresponding to a plurality of second identifiers; the first identification also corresponds to a first side surface of the first detection object, and the second identification also corresponds to a second side surface of the first detection object; the first side face and the second side face are two intersecting side faces in the first detection object.

Based on this, when the first vehicle adopts the monocular camera to collect two side surfaces of the first detection object (that is, two side surfaces including the first detection object in the image to be detected), the first vehicle may mark the passable area boundary of each side surface respectively, and assign different identifiers to the passable area boundaries of different side surfaces, and the first vehicle may determine the passable area boundaries corresponding to the respective side surfaces according to the different identifiers.

With reference to the first aspect, in one possible implementation manner, the ground contact line includes a first ground contact line and a second ground contact line; the first ground contact line is a ground contact line determined by fitting a plurality of boundary points corresponding to the first identifier; the second touchdown line is a touchdown line determined by fitting a plurality of boundary points corresponding to the second identifier.

Based on the above, the first vehicle determines the contact line corresponding to each side face of the first detection object shown in the image to be detected according to the passable area boundary corresponding to each side face. Since the touchdown lines are fitted from points on the boundaries of the passable regions corresponding to each side, the touchdown line corresponding to each side can characterize the partial boundaries of the projection of that side on the ground (i.e., the outermost portions of the projection of each side on the ground).

With reference to the first aspect, in a possible implementation manner, the plurality of boundary points corresponding to the first identifier includes a first boundary point; the first boundary point is the boundary point with the largest distance with the second contact line in the plurality of boundary points with the first identification; the plurality of boundary points corresponding to the second identifier comprise second boundary points; the second boundary point is a boundary point with the largest distance from the first contact line among the plurality of boundary points with the second identifier.

Based on this, the first boundary point can represent a point of the first side surface of the first detection object, which is farthest from the first vehicle, that is, the first boundary point can represent a vertex of the first side surface, which is farthest from the first vehicle. The second boundary point can represent a point of the second side of the first detection object at which the distance from the first vehicle is greatest, that is, the second boundary point can represent a vertex of the second side at which the second side is farthest from the first vehicle.

With reference to the first aspect, in a possible implementation manner, the three-dimensional information of the first detection object is determined according to three points and two lines corresponding to the first detection object; wherein, the first point in the three points is the projection of the first boundary point on the ground; the second point of the three points is the projection of the second boundary point on the ground; the third point of the three points is the second point, and the intersection point of a straight line parallel to the second grounding line and the first grounding line; the first line of the two lines is a connecting line between the first point and the third point; the second line of the two lines is the line between the second point and the third point.

With reference to the first aspect, in a possible implementation manner, a projection of the first boundary point on the ground is determined as a first point; determining a projection of the second boundary point on the ground as a second point; determining the projection of the second boundary point on the ground, wherein the intersection point of a straight line parallel to the second ground contact line and the first ground contact line is a third point; determining a connection line between the first point and the third point as a first line; determining a connection line between the second point and the third point as a second line; determining three-dimensional information of the first detection object from the first point, the second point, the third point, the first line, and the second line.

Based on the technical scheme, the projection of the first boundary point on the ground can represent the point, which is farthest from the projection of the first side surface on the ground and the first vehicle, the projection of the second boundary point on the ground can represent the point, which is farthest from the projection of the second side surface on the ground and the second vehicle, and the third point can represent the intersection point of the projection of the first side surface on the ground and the projection of the second side surface on the ground. The first ground contact line can characterize a direction of projection of the first side on the ground. The second ground contact line can characterize a direction of projection of the second side on the ground. Thus, the first vehicle may determine that the first line is the outermost frame line of the first side projected on the ground and the second line is the outermost frame line of the second side projected on the ground.

Further, the first vehicle may determine the direction of the first detection object according to the direction of the first line and/or the second line and the positions of the first line and the second line in the image to be detected. The first vehicle may determine the size of the first detection object according to the length of the first line and the length of the second line, and the positions of the first line and the second line in the image to be detected, and the first vehicle may determine the position of the first detection object relative to the first vehicle according to the positions of the first line and the second line in the image to be detected.

Therefore, the first vehicle can determine the three-dimensional information of the first detection object only by projecting the specific point on the first detection object according to the passable area boundary and the touch line of the first detection object, and the calculation amount of determining the three-dimensional information of the first detection object by the first vehicle is greatly reduced.

With reference to the first aspect, in one possible implementation manner, the first vehicle determines a first point according to a first boundary point and a first contact line; the first vehicle determines a second point according to the first contact ground wire, the second contact ground wire and the second boundary point; and determining a third point according to the first contact line, the second contact line and the second point.

Based on this, the first vehicle may determine a vertex of the projection of the first detection object on the ground from the passable area boundary and the touchdown line.

With reference to the first aspect, in one possible implementation manner, a first vehicle determines a first straight line; the first straight line is a straight line which is perpendicular to the visual flat line and is a first boundary point in the image to be detected; the first vehicle determines an intersection point of the first straight line and the first ground contact line as a first point.

Based on the method, the first vehicle can quickly and accurately determine the projection of the first boundary point on the ground according to the visual relation between the first contact line and the first boundary point in the image to be detected. The projection of the first boundary point on the ground determined by the first vehicle in this way can further reduce the calculation amount of the three-dimensional information of the first detection object determined by the first vehicle.

With reference to the first aspect, in one possible implementation manner, the first vehicle determines a second straight line and a third straight line; the second straight line is a straight line of an intersection point of the first ground contact line and the visual flat line in the image to be detected and an end point far away from the first ground contact line in the second ground contact line; the third straight line is a straight line which is perpendicular to the visual flat line and is a second boundary point in the image to be detected; the first vehicle determines the intersection of the second line and the third line as a second point.

Based on the method, the first vehicle can quickly and accurately determine the projection of the first boundary point on the ground according to the visual relationship between the first ground contact line and the second ground contact line as well as the visual relationship between the second boundary point in the image to be detected. The projection of the second boundary point on the ground determined by the first vehicle in this way can further reduce the calculation amount of the three-dimensional information of the first detection object determined by the first vehicle.

With reference to the first aspect, in one possible implementation manner, the first vehicle determines a fourth straight line; the fourth straight line is a straight line which is parallel to the second ground contact line and is a second point in the image to be detected; the first vehicle determines an intersection of the fourth straight line and the first ground contact line as a third point.

Based on the above, the first vehicle can quickly and accurately determine the intersection point of the projection of the first side surface and the second side surface on the ground according to the visual relationship of the first ground contact line, the second ground contact line and the second boundary point in the image to be detected. The intersection point of the projection of the first side surface and the second side surface on the ground is determined by the first vehicle in this way, so that the calculation amount of the three-dimensional information of the first detection object determined by the first vehicle can be further reduced.

With reference to the first aspect, in a possible implementation manner, the passable area boundary of the first detection object includes a plurality of boundary points corresponding to the third identifier; the third mark also corresponds to a third side of the first detection object.

Based on this, when the first vehicle adopts the monocular camera to collect one side surface of the first detection object (i.e., one side surface including the first detection object in the image to be detected), the first vehicle may mark the passable area boundary of the side surface and assign a corresponding identifier to the passable area boundary, and the first vehicle may determine the passable area boundary corresponding to the side surface according to the passable area boundary point having the identifier.

With reference to the first aspect, in one possible implementation manner, the ground contact line of the first detection object includes a third ground contact line; the third touchdown line is determined by fitting a plurality of boundary points corresponding to the third identifier.

Based on this, the first vehicle can determine the ground contact line of the side according to the border of the passable area of the side. Since the touchdown line is obtained by fitting points on the boundary of the passable region corresponding to the side face, the touchdown line can represent the partial boundary of the projection of the side face on the ground (i.e. the outermost part of the projection of each face on the ground).

With reference to the first aspect, in a possible implementation manner, the boundary points with the third identifier include a third boundary point and a fourth boundary point; the third boundary point is the farthest point from one end of the third contact line in the plurality of boundary points with the third identification; the fourth boundary point is the farthest point from the other end of the third contact line among the plurality of boundary points having the third identification.

Based on this, the third boundary point and the fourth boundary point can represent two vertexes of the third side surface of the first detection object.

With reference to the first aspect, in a possible implementation manner, the three-dimensional information of the first detection object is determined according to two points and a line corresponding to the first detection object; the first point of the two points is the projection of the third boundary point on the ground; the second of the two points is a projection of the fourth boundary point on the ground.

With reference to the first aspect, in a possible implementation manner, a projection of the third boundary point on the ground is determined as a first point; determining the projection of the fourth boundary point on the ground as a second point; determining a connection line between the first point and the second point as a first line; and determining three-dimensional information of the first detection object according to the first point, the second point and the first line.

Based on the technical scheme, the projection of the third boundary point on the ground can represent a vertex of the projection of the third side surface on the ground. The projection of the fourth boundary point on the ground can represent another top line of the projection of the third side surface on the ground. The connecting line (first line) between the projection of the third boundary point on the ground and the projection of the fourth boundary point on the ground can represent the outermost frame line of the projection of the third side on the ground.

Further, the first vehicle may determine the direction of the first detection object according to the direction of the first line and the position of the first line in the image to be detected. The first vehicle may determine the size of the first detection object according to the length of the first line and the position of the first line in the image to be detected. The first vehicle can determine the position of the first detection object relative to the first vehicle according to the position of the first line in the image to be detected.

With reference to the first aspect, in a possible implementation manner, determining three-dimensional information of the first detection object according to the passable area boundary and the ground contact line includes: determining a first point according to the third boundary point and the third contact line; determining a second point according to the fourth boundary point and the third grounding contact line; from the first point and the second point, a line is determined.

Based on this, the first vehicle determines information of a vertex of the projection of the first detection object on the ground, and an outermost peripheral boundary of the projection of the first detection object on the ground. The first vehicle may determine a size and a direction of the first detection object based on a vertex of the projection of the first detection object on the ground and the outermost peripheral boundary. For example, when the first detection object is the second vehicle, the vertex of the projection of the second vehicle on the ground and the outermost boundary line can represent the size of the second vehicle, and the direction of the outermost boundary line can represent the direction of the second vehicle.

With reference to the first aspect, in a possible implementation manner, determining a first point according to a third boundary point and a third touch ground line includes: determining a fifth straight line; the fifth straight line is a straight line which is perpendicular to the visual flat line and is a third boundary point in the image to be detected; and determining the intersection point of the fifth straight line and the third grounding line as a first point.

Based on this, the first vehicle can quickly and accurately determine the projection of the third boundary point on the ground according to the visual relationship between the third contact line and the third boundary point in the image to be detected. The projection of the third boundary point on the ground determined by the first vehicle in this way can further reduce the calculation amount of the three-dimensional information of the first detection object determined by the first vehicle.

With reference to the first aspect, in a possible implementation manner, determining a second point according to the fourth boundary point and the third ground contact line includes: determining a sixth straight line; the sixth straight line is a straight line which is perpendicular to the visual flat line and is a fourth boundary point in the image to be detected; and determining the intersection point of the sixth straight line and the third grounding line as a second point.

Based on the method, the first vehicle can quickly and accurately determine the projection of the fourth boundary point on the ground according to the visual relationship between the third grounding line and the fourth boundary point in the image to be detected. The projection of the fourth boundary point on the ground determined by the first vehicle in this way can further reduce the calculation amount of the three-dimensional information of the first detection object determined by the first vehicle.

With reference to the first aspect, in a possible implementation manner, the method further includes: inputting the three-dimensional information of the first detection object into the vehicle body coordinate system, and determining at least one of the size, the direction and the relative position of the first detection object.

Based on this, the first vehicle can determine the size, direction, and position of the detection object relative to the first vehicle in the three-dimensional space by inputting the three-dimensional information of the first detection object into the vehicle body coordinate system.

In a second aspect, an apparatus for determining three-dimensional information of a detection object is provided, comprising: a communication unit and a processing unit; the communication unit is used for acquiring an image to be detected; the image to be detected comprises a first detection object; a processing unit for determining a passable area boundary of the first detection object and a touchdown line of the first detection object; the passable area boundary comprises a boundary of the first detection object in the image to be detected; the contact line is a connecting line of an intersection point of the first detection object and the ground; and the processing unit is also used for determining the three-dimensional information of the first detection object according to the passable area boundary and the contact line.

With reference to the second aspect, in one possible implementation manner, the three-dimensional information of the first detection object is used to determine at least one of a size, a direction, and a relative position of the first detection object.

With reference to the second aspect, in a possible implementation manner, the passable area boundary of the first detection object includes a plurality of boundary points corresponding to the first identifier, and boundary points corresponding to a plurality of second identifiers; the first identification also corresponds to a first side surface of the first detection object, and the second identification also corresponds to a second side surface of the first detection object; the first side face and the second side face are two intersecting side faces in the first detection object.

With reference to the second aspect, in one possible implementation manner, the ground contact line includes a first ground contact line and a second ground contact line; the first ground contact line is a ground contact line determined by fitting a plurality of boundary points corresponding to the first identifier; the second touchdown line is a touchdown line determined by fitting a plurality of boundary points corresponding to the second identifier.

With reference to the second aspect, in a possible implementation manner, the boundary points corresponding to the first identifier include a first boundary point; the first boundary point is the boundary point with the largest distance with the second contact line in the plurality of boundary points with the first marks.

The plurality of boundary points corresponding to the second identifier comprise second boundary points; the second boundary point is a boundary point with the largest distance from the first contact line among the plurality of boundary points with the second identifier.

With reference to the second aspect, in a possible implementation manner, the three-dimensional information of the first detection object is determined according to three points and two lines corresponding to the first detection object; wherein, the first point in the three points is the projection of the first boundary point on the ground; the second point of the three points is the projection of the second boundary point on the ground; the third point of the three points is the second point, and the intersection point of a straight line parallel to the second grounding line and the first grounding line; the first line of the two lines is a connecting line between the first point and the third point; the second line of the two lines is the line between the second point and the third point.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a projection of the first boundary point on the ground as a first point; determining a projection of the second boundary point on the ground as a second point; determining the projection of the second boundary point on the ground, wherein the intersection point of a straight line parallel to the second ground contact line and the first ground contact line is a third point; determining a connection line between the first point and the third point as a first line; determining a connection line between the second point and the third point as a second line; determining three-dimensional information of the first detection object from the first point, the second point, the third point, the first line, and the second line.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a first point according to the first boundary point and the first grounding wire; determining a second point according to the first contact ground wire, the second contact ground wire and the second boundary point; and determining a third point according to the first contact line, the second contact line and the second point.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to determine a first straight line; the first straight line is a straight line which is perpendicular to the visual flat line and is a first boundary point in the image to be detected; and determining the intersection point of the first straight line and the first contact line as a first point.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a second straight line and a third straight line; the second straight line is a straight line of an intersection point of the first ground contact line and the visual flat line in the image to be detected and an end point far away from the first ground contact line in the second ground contact line; the third straight line is a straight line which is perpendicular to the visual flat line and is a second boundary point in the image to be detected; and determining the intersection point of the second straight line and the third straight line as a second point.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a fourth straight line; the fourth straight line is a straight line which is parallel to the second ground contact line and is a second point in the image to be detected; and determining the intersection point of the fourth straight line and the first contact line as a third point.

With reference to the second aspect, in a possible implementation manner, the passable area boundary of the first detection object includes a plurality of boundary points corresponding to the third identifier; the third mark also corresponds to a third side of the first detection object.

With reference to the second aspect, in one possible implementation manner, the ground contact line of the first detection object includes a third ground contact line; the third touchdown line is determined by fitting a plurality of boundary points corresponding to the third identifier.

With reference to the second aspect, in a possible implementation manner, the boundary points with the third identifier include a third boundary point and a fourth boundary point; the third boundary point is the farthest point from one end of the third contact line in the plurality of boundary points with the third identification; the fourth boundary point is the farthest point from the other end of the third contact line among the plurality of boundary points having the third identification.

With reference to the second aspect, in a possible implementation manner, the three-dimensional information of the first detection object is determined according to two points and a line corresponding to the first detection object; the first point of the two points is the projection of the third boundary point on the ground; the second of the two points is a projection of the fourth boundary point on the ground.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a projection of the third boundary point on the ground as a first point; determining the projection of the fourth boundary point on the ground as a second point; determining a connection line between the first point and the second point as a first line; and determining three-dimensional information of the first detection object according to the first point, the second point and the first line.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a first point according to the third boundary point and the third contact line; determining a second point according to the fourth boundary point and the third grounding contact line; from the first point and the second point, a line is determined.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a fifth straight line; the fifth straight line is a straight line which is perpendicular to the visual flat line and is a third boundary point in the image to be detected; and determining the intersection point of the fifth straight line and the third grounding line as a first point.

With reference to the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determining a sixth straight line; the sixth straight line is a straight line which is perpendicular to the visual flat line and is a fourth boundary point in the image to be detected; and determining the intersection point of the sixth straight line and the third grounding line as a second point.

With reference to the second aspect, in a possible implementation manner, the processing unit is further configured to: inputting the three-dimensional information of the first detection object into the vehicle body coordinate system, and determining at least one of the size, the direction and the relative position of the first detection object.

In a third aspect, the present application provides an apparatus for determining three-dimensional information of a detection object, comprising: a processor and a memory, wherein the memory is adapted to store a computer program and instructions, and the processor is adapted to execute the computer program and instructions to implement the method as described in the first aspect and any possible implementation manner of the first aspect. The device for determining the three-dimensional information of the detection object may be the first vehicle or a chip in the first vehicle.

In a fourth aspect, the present application provides a smart vehicle comprising: a vehicle body, a monocular camera for acquiring an image to be detected, and a device for determining three-dimensional information of a detection object as described in any one of possible implementations of the second aspect and the second aspect; the apparatus for determining three-dimensional information of a detection object is configured to perform the method for determining three-dimensional information of a detection object as described in the first aspect and any one of the possible implementations of the first aspect, determining three-dimensional information of a detection object.

With reference to the fourth aspect, in one possible implementation manner, the smart vehicle further includes a display screen; the display screen is used for displaying three-dimensional information of the detection object.

In a fifth aspect, the present application provides an Advanced Driver Assistance System (ADAS) comprising the means for determining three-dimensional information of a detection object as described in the second aspect and any one of the possible implementations of the second aspect, the means for determining three-dimensional information of a detection object being configured to perform the method for determining three-dimensional information of a detection object as described in the first aspect and any one of the possible implementations of the first aspect, determining three-dimensional information of a detection object.

In a sixth aspect, the present application provides a computer-readable storage medium having stored therein instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect and any one of the possible implementations of the first aspect.

In a seventh aspect, the present application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method as described in the first aspect and any one of the possible implementations of the first aspect.

It should be appreciated that the description of technical features, solutions, benefits, or similar language in this application does not imply that all of the features and advantages may be realized in any single embodiment. Rather, it is to be understood that the description of a feature or advantage is intended to include the specific features, aspects or advantages in at least one embodiment. Therefore, the descriptions of technical features, technical solutions or advantages in the present specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and advantages described in the present embodiments may also be combined in any suitable manner. One skilled in the relevant art will recognize that an embodiment may be practiced without one or more of the specific features, aspects, or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments.

Drawings

FIG. 1 is a first schematic structural diagram of a vehicle according to an embodiment of the present disclosure;

fig. 2 is a system architecture diagram of an ADAS system according to an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a computer system according to an embodiment of the present disclosure;

fig. 4 is a first schematic view of an application of a cloud-side command autonomous driving vehicle according to an embodiment of the present disclosure;

fig. 5 is a schematic application diagram of a cloud-side command autonomous driving vehicle according to an embodiment of the present disclosure;

FIG. 6 is a block diagram of a computer program product according to an embodiment of the present application;

fig. 7 is a schematic flowchart of a method for determining three-dimensional information of a detection object according to an embodiment of the present application;

FIG. 8a is a schematic diagram of a first detection object according to an embodiment of the present disclosure;

FIG. 8b is a schematic view of another first test object provided in an embodiment of the present application;

fig. 9 is a schematic flowchart of another method for determining three-dimensional information of a detection object according to an embodiment of the present application;

fig. 10 is a schematic flowchart of another method for determining three-dimensional information of a detection object according to an embodiment of the present application;

fig. 11 is a schematic diagram of three-dimensional information of a first detection object according to an embodiment of the present disclosure;

fig. 12 is a schematic flowchart of another method for determining three-dimensional information of a detection object according to an embodiment of the present application;

fig. 13 is a schematic diagram of three-dimensional information of another first detection object according to an embodiment of the present disclosure;

fig. 14 is a schematic structural diagram of an apparatus for determining three-dimensional information of a detection object according to an embodiment of the present application.

Detailed Description

In the description of this application, "/" means "or" unless otherwise stated, for example, A/B may mean A or B. "and/or" herein is merely an association describing an associated object, and means that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. Further, "at least one" means one or more, "a plurality" means two or more. The terms "first", "second", and the like do not necessarily limit the number and execution order, and the terms "first", "second", and the like do not necessarily limit the difference.

It is noted that, in the present application, words such as "exemplary" or "for example" are used to mean exemplary, illustrative, or descriptive. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

The embodiment of the application provides a method and a device for determining three-dimensional information of a detection object, and the method is applied to a vehicle or other equipment (such as a cloud server, a mobile phone terminal and the like) with a function of controlling the vehicle. The vehicle or other equipment can implement the method for determining the three-dimensional information of the detection object provided by the embodiment of the application through the contained components (including hardware and software), the vehicle acquires the image to be detected according to the image acquisition device, and determines the three-dimensional information (size, direction and relative position) of the detection object, so that the vehicle can plan the driving path of the vehicle according to the three-dimensional information of the detection object.

Fig. 1 is a functional block diagram of a vehicle 100 provided in an embodiment of the present application, where the vehicle 100 may be an intelligent vehicle. In one embodiment, the vehicle 100 determines the three-dimensional information of the detection objects according to the image to be detected acquired by the image acquisition device, so that the vehicle can plan the driving path of the vehicle according to the three-dimensional information of the detection objects.

Vehicle 100 may include various subsystems such as a travel system 110, a sensor system 120, a control system 130, one or more peripherals 140, as well as a power supply 150, a computer system 160, and a user interface 170. Alternatively, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements. In addition, each of the sub-systems and elements of the vehicle 100 may be interconnected by wire or wirelessly.

The travel system 110 may include components that provide powered motion to the vehicle 100. In one embodiment, the travel system 110 may include an engine 111, a transmission 112, an energy source 113, and wheels 114. The engine 111 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 111 converts the energy source 113 into mechanical energy.

Examples of energy sources 113 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 113 may also provide energy to other systems of the vehicle 100.

The transmission 112 may transmit mechanical power from the engine 111 to the wheels 114. The transmission 112 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 112 may also include other devices, such as a clutch. Wherein the drive shaft may comprise one or more shafts that may be coupled to one or more wheels 114.

The sensor system 120 may include several sensors that sense information about the environment surrounding the vehicle 100. For example, the sensor system 120 may include a positioning system 121 (the positioning system may be a Global Positioning System (GPS), a Beidou system, or other positioning system), an Inertial Measurement Unit (IMU) 122, a radar 123, a laser radar 124, and a camera 125. The sensor system 120 may also include sensors that monitor internal systems of the vehicle 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors may be used to detect the object and its corresponding characteristics (position, shape, orientation, velocity, etc.). Such detection and identification is a key function of the safe operation of the automatic driving of the vehicle 100.

The positioning system 121 may be used to estimate the geographic location of the vehicle 100. The IMU 122 is used to sense position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 122 may be a combination of an accelerometer and a gyroscope.

The radar 123 may utilize radio signals to sense objects within the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, radar 123 may also be used to sense the speed and/or heading of an object.

Lidar 124 may utilize a laser to sense objects in the environment in which vehicle 100 is located. In some embodiments, lidar 124 may include one or more laser sources, laser scanners, and one or more detectors, among other system components.

The camera 125 may be used to capture multiple images of the surroundings of the vehicle 100, as well as multiple images within the vehicle cabin. The camera 125 may be a still camera or a video camera. The control system 130 may control the operation of the vehicle 100 and its components. Control system 130 may include various elements including a steering system 131, a throttle 132, a braking unit 133, a computer vision system 134, a route control system 135, and an obstacle avoidance system 136.

The steering system 131 is operable to adjust the heading of the vehicle 100. For example, in one embodiment, a steering wheel system.

The throttle 132 is used to control the operating speed of the engine 111 and thus the speed of the vehicle 100.

The brake unit 133 is used to control the vehicle 100 to decelerate. The brake unit 133 may use friction to slow the wheel 114. In other embodiments, the brake unit 133 may convert the kinetic energy of the wheel 114 into an electrical current. The brake unit 133 may take other forms to slow the rotational speed of the wheels 114 to control the speed of the vehicle 100.

The computer vision system 134 may be operable to process and analyze images captured by the camera 125 to identify objects and/or features in the environment surrounding the vehicle 100 as well as limb and facial features of a driver within the vehicle cabin. The objects and/or features may include traffic signals, road conditions, and obstacles, and the limb and facial features of the driver include the driver's behavior, line of sight, expression, and the like. The computer vision system 134 may use object recognition algorithms, motion from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 134 may be used to map an environment, track objects, estimate the speed of objects, determine driver behavior, face recognition, and so forth.

The route control system 135 is used to determine a travel route of the vehicle 100. In some embodiments, route control system 135 may combine data from sensors, positioning system 121, and one or more predetermined maps to determine a travel route for vehicle 100.

Obstacle avoidance system 136 is used to identify, assess, and avoid or otherwise negotiate potential obstacles in the environment of vehicle 100.

Of course, in one example, the control system 130 may add portions of components not shown above; or replacing some of the components shown above with other components; or alternatively some of the components shown above may be reduced.

Vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through peripherals 140. The peripheral devices 140 may include a wireless communication system 141, an in-vehicle computer 142, a microphone 143, and/or a speaker 144.

In some embodiments, the peripheral device 140 provides a means for a user of the vehicle 100 to interact with the user interface 170. For example, the in-vehicle computer 142 may provide information to a user of the vehicle 100. The user interface 170 may also operate the in-vehicle computer 142 to receive user input. The in-vehicle computer 142 may be operated through a touch screen. In other cases, the peripheral device 140 may provide a means for the vehicle 100 to communicate with other devices located within the vehicle. For example, the microphone 143 may receive audio (e.g., voice commands or other audio input) from a user of the vehicle 100. Similarly, the speaker 144 may output audio to a user of the vehicle 100.

Wireless communication system 141 may wirelessly communicate with one or more devices directly or via a communication network. For example, wireless communication system 141 may use 3G cellular communication, such as CDMA, EVD0, GSM/GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 141 may communicate with a Wireless Local Area Network (WLAN) using WiFi. In some embodiments, the wireless communication system 141 may utilize an infrared link, Bluetooth, or ZigBee to communicate directly with the devices. Wireless communication system 141 may also communicate with devices using other wireless protocols. Such as various vehicle communication systems. The wireless communication system 141 may include one or more Dedicated Short Range Communications (DSRC) devices.

The power supply 150 may provide power to various components of the vehicle 100. In one embodiment, power source 150 may be a rechargeable lithium ion or lead acid battery. One or more battery packs of such batteries may be configured as a power source to provide power to various components of the vehicle 100. In some embodiments, power source 150 and energy source 113 may be implemented together, such as a pure electric vehicle or a hybrid electric vehicle in a new energy vehicle, or the like.

Some or all of the functions of vehicle 100 are controlled by computer system 160. The computer system 160 may include at least one processor 161, the processor 161 executing instructions 1621 stored in a non-transitory computer readable medium, such as the data storage device 162. The computer system 160 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

Processor 161 may be any conventional processor, such as a commercially available Central Processing Unit (CPU). Alternatively, the processor may be a dedicated device such as an Application Specific Integrated Circuit (ASIC) or other hardware-based processor. Although fig. 1 functionally illustrates a processor, memory, and other elements within the same physical housing, those skilled in the art will appreciate that the processor, computer system, or memory may actually comprise multiple processors, computer systems, or memories that may or may not be stored within the same physical housing. For example, the memory may be a hard drive, or other storage medium located in a different physical enclosure. Thus, references to a processor or computer system are to be understood as including references to a collection of processors or computer systems or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described herein, some components, such as the steering component and the retarding component, may each have their own processor that performs only computations related to the component-specific functions.

In various aspects described herein, the processor may be located in a device remote from and in wireless communication with the vehicle. In other aspects, some of the processes described herein are executed on a processor disposed within the vehicle and others are executed by a remote processor, including taking the steps necessary to perform a single maneuver.

In some embodiments, the data storage device 162 may include instructions 1621 (e.g., program logic), which instructions 1621 may be executed by the processor 161 to perform various functions of the vehicle 100, including all or part of the functions described above. The data storage device 162 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and/or control one or more of the travel system 110, the sensor system 120, the control system 130, and the peripheral devices 140.

In addition to instructions 1621, data storage device 162 may also store data such as road maps, route information, the location, direction, speed, and other such vehicle data of the vehicle, as well as other information. Such information may be used by vehicle 100 and computer system 160 during operation of vehicle 100 in autonomous, semi-autonomous, and/or manual modes.

For example, in one possible embodiment, the data storage device 162 may obtain information about obstacles in the surrounding environment, such as the positions of other vehicles, road edges, greenbelts, and the like, the distance between the obstacles and the vehicle, and the distance between the obstacles, which are obtained by the vehicle based on the sensors in the sensor system 120. The data storage device 162 may also obtain environmental information from the sensor system 120 or other components of the vehicle 100, such as whether a green belt, a lane, a pedestrian, etc. is present near the environment in which the vehicle is currently located, or whether a green belt, a pedestrian, etc. is present near the environment in which the vehicle is currently located as calculated by a machine learning algorithm. In addition to the above, the data storage device 162 may store the status information of the vehicle itself, including but not limited to the position, speed, acceleration, heading angle, etc. of the vehicle, as well as the status information of other vehicles with which the vehicle interacts. As such, the processor 161 may acquire such information from the data storage device 162, and determine a passable area of the vehicle based on environmental information of an environment in which the vehicle is located, state information of the vehicle itself, state information of other vehicles, and the like, and determine a final driving strategy based on the passable area to control the vehicle 100 to autonomously drive.

A user interface 170 for providing information to or receiving information from a user of the vehicle 100. Optionally, the user interface 170 may interact with one or more input/output devices within the set of peripheral devices 140, such as one or more of the wireless communication system 141, the in-vehicle computer 142, the microphone 143, and the speaker 144.

The computer system 160 may control the vehicle 100 based on information obtained from various subsystems (e.g., the travel system 110, the sensor system 120, and the control system 130) and information received from the user interface 170. For example, computer system 160 may control steering system 131 to alter the vehicle heading to avoid obstacles detected by sensor system 120 and obstacle avoidance system 136 based on information from control system 130. In some embodiments, the computer system 160 may control many aspects of the vehicle 100 and its subsystems.

Alternatively, one or more of these components described above may be mounted or associated separately from the vehicle 100. For example, the data storage device 162 may exist partially or completely separate from the vehicle 100. The above components may be coupled together for communication by wired and/or wireless means.

Optionally, the above components are only an example, in an actual application, components in the above modules may be added or deleted according to an actual need, and fig. 1 should not be construed as limiting the embodiment of the present application.

An autonomous automobile traveling on a roadway, such as vehicle 100 above, may determine an adjustment command for the current speed based on other vehicles within its surrounding environment. The objects in the environment around the vehicle 100 may be traffic control devices, or other types of objects such as green belts, among others. In some examples, each object within the surrounding environment may be considered independently, and the speed adjustment instructions for the vehicle 100 may be determined based on respective characteristics of the object, such as its current speed, acceleration, separation from the vehicle, and so forth.

Alternatively, vehicle 100, or a computer device associated therewith (e.g., computer system 160, computer vision system 134, data storage 162 of fig. 1), which is an autonomous automobile, may derive a state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.) based on the identified measurement data and determine a relative position of an obstacle in the surrounding environment to the vehicle at the current time. Alternatively, the boundaries of the passable areas formed by each obstacle are dependent on each other, and therefore, it is also possible to determine the boundaries of the passable areas of the vehicle together with all the acquired measurement data, removing the areas of the passable areas that are not actually passable. The vehicle 100 is able to adjust its driving strategy based on the detected passable areas of the vehicle. In other words, the autonomous vehicle is able to determine what steady state the vehicle needs to adjust to (e.g., accelerate, decelerate, turn, or stop, etc.) based on the detected passable area of the vehicle. In this process, other factors may also be considered to determine the speed adjustment command for the vehicle 100, such as the lateral position of the vehicle 100 in the road being traveled, the curvature of the road, the proximity of static and dynamic objects, and so forth.

In addition to providing instructions to adjust the speed of the autonomous vehicle, the computer device may also provide instructions to modify the steering angle of the vehicle 100 to cause the autonomous vehicle to follow a given trajectory and/or to maintain a safe lateral and longitudinal distance of the autonomous vehicle from nearby objects (e.g., cars in adjacent lanes).

The vehicle 100 may be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, an amusement car, a playground vehicle, construction equipment, a trolley, a golf cart, a train, a trolley, etc., and the embodiment of the present invention is not particularly limited.

In other embodiments of the present application, the autonomous vehicle may further include a hardware structure and/or a software module, and the functions described above are implemented in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether any of the above-described functions is implemented as a hardware structure, a software module, or a hardware structure plus a software module depends upon the particular application and design constraints imposed on the technical solution.

In one implementation, referring to fig. 2, the method for determining three-dimensional information of a detection object provided in the embodiment of the present application is applied to an ADAS system 200 shown in fig. 2. As shown in fig. 2, the ADAS system 200 includes a hardware system 201, a perception fusion system 202, a planning system 203, and a control system 204.

The hardware system 201 is configured to collect road information, vehicle information, obstacle information, and the like around the first vehicle. The currently commonly used hardware system 201 mainly includes a camera, a video capture card, and the like. In the embodiment of the present application, the hardware system 201 includes a monocular camera.

The perception fusion system 202 is configured to process the image information collected by the hardware system 201, and determine target information (including vehicle information, pedestrian information, red road information, obstacle information, and the like) around the first vehicle and road structure information (including lane line information, road edge information, and the like).

The planning system 203 is configured to plan a driving route, a driving speed, and the like of the first vehicle according to the target information and the road structure information, and generate planning information.

The control system 204 is configured to convert the planning information generated by the planning system 203 into control information, and issue the control information to the first vehicle, so that the first vehicle travels along the traveling route and the traveling speed of the first vehicle planned by the planning system 30 according to the control information.

And the vehicle-mounted communication module 205 (not shown in fig. 2) is used for information interaction between the own vehicle and other vehicles.

The storage component 206 (not shown in fig. 2) is configured to store executable codes of the above modules, and execute the executable codes to implement part or all of the method flows of the embodiments of the present application.

In one possible implementation of the embodiments of the present application, as shown in fig. 3, the computer system 160 shown in fig. 1 includes a processor 301, the processor 301 is coupled to a system bus 302, and the processor 301 may be one or more processors, each of which may include one or more processor cores. A display adapter 303 may drive a display 324, the display 324 coupled with system bus 302. System BUS 302 is coupled via a BUS bridge 304 to an input/output (I/O) BUS (BUS)305, an I/O interface 306 coupled to I/O BUS 305, and I/O interface 306 in communication with various I/O devices, such as an input device 307 (e.g., keyboard, mouse, touch screen, etc.), multimedia disk (media tray)308, (e.g., CD-ROM, multimedia interface, etc.). A transceiver 309 (which can send and/or receive radio communication signals), a camera 310 (which can capture still and motion digital video images), and an external Universal Serial Bus (USB) port 311. Wherein, optionally, the interface connected with the I/O interface 306 may be a USB interface.

The processor 301 may be any conventional processor, including a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, or a combination thereof. Alternatively, the processor 301 may also be a dedicated device such as an application specific integrated circuit ASIC. Alternatively, the processor 301 may also be a neural network processor or a combination of a neural network processor and the conventional processor described above.

Alternatively, in various embodiments of the present application, the computer system 160 may be located remotely from the smart vehicle and communicate wirelessly with the smart vehicle 100. In other aspects, some processes of the present application may be provided for execution on a processor within a smart vehicle, with other processes being performed by a remote processor, including taking the actions necessary to perform a single maneuver.

Computer system 160 may communicate with a software deploying server (deploying server)313 via a network interface 312. Alternatively, the network interface 312 may be a hardware network interface, such as a network card. The Network (Network)314 may be an external Network, such as the internet, or an internal Network, such as an ethernet or a Virtual Private Network (VPN), and optionally, the Network314 may also be a wireless Network, such as a WiFi Network, a cellular Network, and so on.

Hard drive interface 315 is coupled to system bus 302. Hard drive interface 315 is coupled to hard drive 316. A system memory 317 is coupled to system bus 302. Data running in system memory 317 may include an Operating System (OS)318 and application programs 319 of computer system 160.

The Operating System (OS)318 includes, but is not limited to, a Shell 320 and a kernel 321. Shell 320 is an interface between the user and kernel 321 of operating system 318. Shell 320 is the outermost tier of operating system 318. The shell manages the interaction between users and the operating system 318: awaits user input, interprets the user input to the operating system 318, and processes the output results of the various operating systems 318.

The kernel 321 is made up of the portion of the operating system 318 that manages memory, files, peripherals, and system resources, interacting directly with the hardware. The kernel 321 of the operating system 318 typically runs processes and provides inter-process communication, CPU slot management, interrupts, memory management, IO management, and the like.

The application programs 319 include autopilot-related programs 323 such as programs for managing the interaction of an autonomous vehicle with on-road obstacles, programs for controlling the travel route or speed of an autonomous vehicle, programs for controlling the interaction of an autonomous vehicle with other on-road vehicles/autonomous vehicles, and the like. Application 319 also resides on the system of the deploying server 313. In one embodiment, computer system 160 may download application 319 from deploying server 313 when application 319 needs to be executed.

As another example, the application 319 can be an application that controls a vehicle to determine a driving strategy based on the passable area of the vehicle and a conventional control module as described above. The processor 301 of the computer system 160 invokes the application 319 to obtain the driving strategy.

Sensor 322 is associated with computer system 160. The sensors 322 are used to detect the environment surrounding the computer system 160. For example, the sensors 322 may detect animals, cars, obstacles, and/or crosswalks, etc. Further sensors 322 may also detect the environment surrounding such objects as animals, cars, obstacles, and/or crosswalks. Such as: the environment surrounding the animal, for example, other animals present around the animal, weather conditions, brightness of the environment surrounding the animal, and the like. Alternatively, if the computer system 160 is located on an autonomous vehicle, the sensor 322 may be at least one of a camera, an infrared sensor, a chemical detector, a microphone, and the like.

In other embodiments of the present application, computer system 160 may also receive information from, or transfer information to, other computer systems. Alternatively, sensor data collected from the sensor system 120 of the vehicle 100 may be transferred to another computer, where it is processed. As shown in fig. 4, data from computer system 160 may be transmitted via a network to cloud-side computer system 410 for further processing. The network and intermediate nodes may comprise various configurations and protocols, including the internet, world wide web, intranets, virtual private networks, wide area networks, local area networks, private networks using proprietary communication protocols of one or more companies, ethernet, WiFi, and HTTP, as well as various combinations of the foregoing. Such communication may be performed by any device capable of communicating data to and from other computers, such as modems and wireless interfaces.

In one example, computer system 410 may include a server having multiple computers, such as a load balancing server farm. To receive, process, and transmit data from computer system 160, server 420 exchanges information with various nodes of the network. The computer system 410 may have a configuration similar to computer system 160 and have a processor 430, memory 440, instructions 450, and data 460.

In one example, the data 460 of the server 420 may include providing weather-related information. For example, server 420 may receive, monitor, store, update, and transmit various information related to target objects in the surrounding environment. This information may include target category, target shape information, and target tracking information, for example, in a report form, radar information form, forecast form, and the like.

Referring to fig. 5, an example of autonomous driving vehicle and cloud service center (cloud server) interaction. The cloud service center may receive information (such as vehicle sensors collecting data or other information) from vehicles 513, 512 within its operating environment 500 via a network 511, such as a wireless communication network. Among them, the vehicle 513 and the vehicle 512 may be smart vehicles.

The cloud service center 520 operates the stored programs related to controlling the automatic driving of the automobile according to the received data to control the vehicles 513 and 512. The programs related to controlling the automatic driving of the automobile can be as follows: a program for managing the interaction of the autonomous vehicle with obstacles on the road, or a program for controlling the route or speed of the autonomous vehicle, or a program for controlling the interaction of the autonomous vehicle with other autonomous vehicles on the road.

For example, cloud service center 520 may provide portions of a map to vehicles 513, 512 via network 511. In other examples, operations may be divided among different locations. For example, multiple cloud service centers may receive, validate, combine, and/or send information reports. Information reports and/or sensor data may also be sent between vehicles in some examples. Other configurations are also possible.

In some examples, the cloud service center 520 sends suggested solutions to the smart vehicle regarding possible driving conditions within the environment (e.g., informing of a front obstacle and informing of how to bypass it)). For example, cloud service center 520 may assist a vehicle in determining how to proceed when facing a particular obstacle within the environment. The cloud service center 520 sends a response to the smart vehicle indicating how the vehicle should travel in the given scenario. For example, the cloud service center 520 may confirm the presence of a temporary stop sign ahead of the road based on the collected sensor data, and determine that the lane is closed due to construction, for example, based on a "lane closure" sign and sensor data of construction vehicles. Accordingly, the cloud service center 520 sends a suggested operating mode for the vehicle to pass the obstacle (e.g., instructing the vehicle to change lanes on another road). The cloud service center 520 observes the video stream within its operating environment 500 and, having confirmed that the smart vehicle can safely and successfully traverse obstacles, the operational steps used for the smart vehicle may be added to the driving information map. Accordingly, this information may be sent to other vehicles in the area that may encounter the same obstacle in order to assist the other vehicles not only in recognizing the closed lane but also in knowing how to pass.

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage medium or other non-transitory medium or article of manufacture in a machine-readable format. Fig. 6 schematically illustrates a conceptual partial view of an example computer program product comprising a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein. In one embodiment, the example computer program product 600 is provided using a signal bearing medium 601. The signal bearing medium 601 may include one or more program instructions 602 that, when executed by one or more processors, may provide all or part of the functionality described above with respect to fig. 2-5, or may provide all or part of the functionality described in subsequent embodiments. For example, referring to the embodiment shown in fig. 7, one or more features of S101-S103 may be undertaken by one or more instructions associated with the signal bearing medium 601. Further, program instructions 602 in FIG. 6 also describe example instructions.

In some examples, signal bearing medium 601 may include a computer readable medium 603 such as, but not limited to, a hard disk drive, a Compact Disc (CD), a Digital Video Disc (DVD), a digital tape, a Memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like. In some implementations, the signal bearing medium 601 may include a computer recordable medium 604 such as, but not limited to, a memory, a read/write (R/W) CD, a R/W DVD, and so forth. In some implementations, the signal bearing medium 601 may include a communication medium 605 such as, but not limited to, a digital and/or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.). Thus, for example, the signal bearing medium 601 may be conveyed by a wireless form of communication medium 605 (e.g., a wireless communication medium that conforms to the IEEE 802.11 standard or other transmission protocol). The one or more program instructions 602 may be, for example, computer-executable instructions or logic-implementing instructions. In some examples, a computing device such as described with respect to fig. 2-6 may be configured to provide various operations, functions, or actions in response to program instructions 602 communicated to the computing device by one or more of a computer readable medium 603, and/or a computer recordable medium 604, and/or a communications medium 605. It should be understood that the arrangements described herein are for illustrative purposes only. Thus, those skilled in the art will appreciate that other arrangements and other elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used instead, and that some elements may be omitted altogether depending upon the desired results. In addition, many of the described elements are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, in any suitable combination and location.

The above description briefly describes an application scenario of the method for determining three-dimensional information of a detection target described in the embodiments of the present application.

In order to make the present application clearer, a brief description of some concepts related to the present application will be given below.

1. Contact ground wire

The contact line is a line segment formed by points of actual contact between a detection object in an image to be detected and the ground.

Taking the detection object as a vehicle as an example, the ground contact line of the vehicle is a connecting line between a tire of the vehicle and a ground contact point. The ground contact line of a vehicle may be distinguished by four sides of the vehicle (left, right, front, and rear sides, respectively). One side of the vehicle corresponds to a grounding wire.

Specifically, taking a common household car as an example, a contact point between a tire at the left front of the vehicle and the ground is marked as a contact point 1; the contact point of the right front tire of the vehicle and the ground is marked as a contact point 2; the contact point of the left rear tire of the vehicle and the ground is marked as a contact point 3; the contact point of the rear right tire of the vehicle with the ground is denoted as contact point 4.

The corresponding ground contact line on the left side of the vehicle is the connecting line between the contact point 1 and the contact point 3.

The ground contact line corresponding to the right side of the vehicle is a connecting line between the contact point 2 and the contact point 4.

The ground contact line corresponding to the front side of the vehicle is a connecting line between the contact point 1 and the contact point 2.

The ground contact line corresponding to the rear side of the vehicle is a connection line between the contact point 3 and the contact point 4.

It should be noted that the portion of the vehicle that contacts the ground, typically a contact surface (which may be considered approximately rectangular). In the application, the vertex of the left front part of the contact surface between the tire at the left front part of the vehicle and the ground can be used as a contact point 1; the vertex of the right front of the contact surface of the right front tire of the vehicle and the ground is taken as a contact point 2; taking the vertex of the left rear part of the contact surface between the tire at the left rear part of the vehicle and the ground as a contact point 3; the contact point 4 is the right rear vertex of the contact surface of the vehicle right rear tire with the ground.

2. Neural network model

The neural network model is an information processing system formed by connecting a large number of processing units (marked as neurons) with each other, and the neurons in the neural network model contain corresponding mathematical expressions. After the data is input into the neuron, the neuron runs the included mathematical expression, calculates the input data and generates output data. Wherein the input data of each neuron is the output data of the last neuron connected with the neuron; the output data of each neuron is the input data of the next neuron connected to it.

In the neural network model, after data is input, the neural network model selects corresponding neurons for the input data according to self learning training, calculates the input data according to the neurons, and determines and outputs a final operation result. Meanwhile, the neural network can continuously learn and evolve in the data operation process, the operation process of the neural network is continuously optimized according to the feedback of the operation result, the more times of operation training of the neural network model are, the more the obtained result is fed back, and the more accurate the calculation result is.

The neural network model described in the embodiment of the present application is used for processing the picture acquired by the image acquisition device and determining the passable area boundary (marked as the passable area boundary of the detection object) on each detection object in the image.

3. Passable area (freespace)

The passable area refers to an area through which a vehicle can travel. For example, a pedestrian, an obstacle, and an open area between other vehicles in a front area detected by the first vehicle is denoted as a passable area of the first vehicle.

The passable area is generally located on the boundary of the detection object, so in this embodiment of the application, the passable area located on the boundary of the detection object may be used to represent the boundary of the first detection object, and then the three-dimensional information of the first detection object is determined according to the passable area located on the first detection object.

In the embodiment of the present application, the passable region boundary of the detection object output by the neural network model is generally shown by a plurality of points with corresponding identifications. For example, the passable area on the left side of the second vehicle includes a plurality of points located on boundary points on the left side of the vehicle. The plurality of points are used to characterize a passable area to the left of the second vehicle.

Furthermore, of the points output by the neural network model, points located on different sides of the detected object may have different identifications, which may include, for example: the mark '00' is used for representing a point positioned on the boundary of the passable area on the left side of the detection object; the mark '01' is used for representing the mark of a point positioned on the border of the passable area on the right side of the detection object; an identifier "10" for characterizing an identifier of a point located on the border of the passable area on the front side of the detection object; the designation "11" is used to characterize the designation of the points located on the border of the passable area on the rear side of the vehicle.

4. Horizon line

The horizon refers to a straight line in the image parallel to the line of sight. In the embodiment of the present application, the horizon refers to a straight line in the image which is the same as the height of the image acquisition device and is parallel to the image acquisition device.

5. Vanishing point (vanising point)

According to the perspective principle of the image, two parallel straight lines on the horizontal plane intersect at a point at the visual plane in the two-dimensional image, and the intersection point is a vanishing point.

6. Vehicle body coordinate system

The vehicle body coordinate system refers to a three-dimensional coordinate system with the origin of the coordinate system located on the vehicle body. Generally, the origin of the vehicle body coordinate system coincides with the center of mass of the vehicle, the X-axis is directed directly in front of the vehicle in the vehicle length direction, the Y-axis is directed in the direction to the left of the driver in the vehicle width direction, and the Z-axis is directed above the vehicle in the vehicle height direction.

The foregoing is a brief introduction to some of the matters and concepts related to this application.

Currently, in order to determine three-dimensional information of a second vehicle around a first vehicle, the following three methods of determining vehicle information are proposed. Respectively as follows: method 1, vehicle 2D detection, method 2, vehicle binocular 3D detection, and method 3, vehicle laser point cloud detection. Hereinafter, method 1, method 2, and method 3 will be described in detail.

Method 1, vehicle 2D detection

Vehicle 2D detection is: the first vehicle determines image information displayed by the second vehicle in the image to be detected; the first vehicle frames out image information displayed in the image to be detected by the second vehicle in a rectangular frame form; and the first vehicle calculates the distance between the second vehicle and the first vehicle according to the position of the lower edge of the rectangular frame, and determines the relative position of the second vehicle and the first vehicle.

It follows that in this method, the first vehicle can only determine the position information of the second vehicle relative to the first vehicle. However, in the intelligent driving scenario, the vehicle needs to determine the size, direction, etc. of the vehicle in addition to the position information of the vehicle, so as to determine whether the surrounding vehicle interferes with the driving of the vehicle.

Therefore, the requirement of the first vehicle for other vehicle information in an intelligent driving scene cannot be met by means of simple vehicle 2D detection.

Method 2, binocular 3D detection of vehicle

Binocular 3D detection depth information of a detection object can be obtained by determining a difference between images of the detection object acquired by two image acquisition devices located at different positions. By establishing the corresponding relation of the same point in the image, the mapping points of the same space physical point in different images are corresponded to form a parallax (Disparity) image, and the 3D information of the detection object can be determined according to the parallax image.

The binocular 3D detection of the vehicle is: the first vehicle adopts a binocular camera and collects two images of the second vehicle from two angles respectively. The first vehicle calculates the three-dimensional information of the second vehicle according to the position deviation of the same point on the second vehicle in the two images. The size, the direction and the position information relative to the first vehicle of the second vehicle can be accurately calculated through a binocular 3D detection algorithm.

However, hardware equipment of the binocular camera is expensive, the manufacturing requirement of the binocular camera applied to the intelligent vehicle is high, and the algorithm used in the current process of determining the 3D information of the second vehicle needs high image labeling cost and calculation amount.

Method 3, vehicle laser point cloud detection

When a laser beam irradiates the surface of an object, the reflected laser beam carries information such as direction, distance and the like. When the laser beam is scanned along a certain trajectory, the reflected laser spot information is recorded while scanning, and since the scanning is extremely fine, a large number of laser spots can be obtained, and a laser point cloud can be formed.

The vehicle laser point cloud detection is as follows: the first vehicle emits laser light to the surroundings and scans the surrounding detection object. The first vehicle receives laser point cloud data returned by surrounding detection objects, and the point cloud data comprises point cloud data returned by a second vehicle and point cloud data returned by other detection objects. The first vehicle adopts algorithms such as machine learning or deep learning and the like to map the laser point cloud data returned by the second vehicle into a certain data structure. And extracting each point or characteristic from the data by the first vehicle, clustering the point cloud data according to the characteristics, and classifying similar point clouds into one class. And the first vehicle inputs the clustered point cloud into a corresponding classifier for classification and identification, and point cloud data of a second vehicle is determined. The first vehicle maps the point cloud data of the second vehicle back to the three-dimensional point cloud data, a 3D surrounding frame of the second vehicle is constructed, and three-dimensional information of the second vehicle is determined.

Although vehicle laser point cloud detection has good detection precision, the hardware cost of a laser radar is high, the data calculation amount of point cloud data is large, a large amount of calculation resources need to be consumed, and a large amount of GPU resources of a first vehicle are occupied.

In order to solve the problems that in the prior art, when a first vehicle determines two-dimensional information of a second vehicle, the size and direction information of the second vehicle cannot be accurately determined, and when the first vehicle determines three-dimensional information of the second vehicle, binocular 3D detection of the vehicle or laser point cloud detection of the vehicle is adopted, hardware cost is high, and calculation complexity is high. The embodiment of the application provides a method for determining three-dimensional information of a detection object, wherein a first vehicle can determine a passable area boundary and a grounding line of the first detection object according to an acquired image to be detected, and further, the first vehicle determines the three-dimensional information represented by the first detection object in a two-dimensional image according to the passable area boundary and the grounding line of the first detection object.

The image to be detected can be a two-dimensional image collected by a monocular camera. Therefore, the first vehicle can determine the three-dimensional information of the first detection object by acquiring the image information of the first detection object through the monocular camera. Compared with the method that the first vehicle needs to rely on a binocular camera to acquire the image information of the first detection object to determine the three-dimensional information of the first detection object or the method that the first vehicle relies on a laser radar to determine the three-dimensional information of the first detection object in the prior art, the method that the first vehicle adopts a monocular camera to acquire the image information of the first detection object to determine the three-dimensional information of the first detection object in the application can greatly reduce the hardware cost for determining the three-dimensional information of the first detection object.

In addition, according to the method for determining the three-dimensional information of the detection object, the first vehicle only needs to mark the passable area boundary of the first detection object, and the contact line of the first detection object is determined according to the passable area boundary of the first detection object. The first vehicle can determine the three-dimensional information represented by the first detection object in the two-dimensional image according to the passable area boundary and the contact line of the first detection object and by combining the visual relationship and the like of the first detection object in the image to be detected. Therefore, the method for determining the three-dimensional information of the detection object does not need the first vehicle to perform other additional data annotation training, so that the calculation amount for determining the three-dimensional information of the first detection object is reduced, and the GPU resources occupied by determining the three-dimensional information of the first detection object are reduced.

Hereinafter, a method for determining three-dimensional information of a detection object provided in an embodiment of the present application will be described in detail, and as shown in fig. 7, the method includes:

s101, the first vehicle acquires an image to be detected.

The image to be detected comprises a first detection object.

In the field of intelligent driving, the detection objects can be vehicles, pedestrians, obstacles and the like. In the embodiment of the present application, a description will be given taking a detection target as a second vehicle as an example.

The image to be detected can be a picture acquired by a vehicle-mounted image acquisition device. On-board image capture devices are commonly used to capture other vehicles located in front of the vehicle; alternatively, the vehicle-mounted image acquisition device can acquire 360-degree all-around images of the vehicle to obtain information of all other vehicles around the vehicle.

It should be noted that the image capturing device described in the embodiment of the present application may be a monocular camera, and when the first vehicle executes the method for determining the three-dimensional information of the detection object provided in the embodiment of the present application, the first vehicle may be an in-vehicle terminal device provided in the first vehicle, or another device having a data processing capability.

S102, the first vehicle determines the border of the passable area of the first detection object and the contact line of the first detection object.

The border of the passable area of the first detection object comprises the border of the first detection object in the image to be detected. The grounding contact line of the first detection object is a connection line of the intersection point of the first detection object and the ground.

The passable area boundary of the first detection object is the passable area boundary of the first detection object output by the neural network model after the image to be detected is input to the neural network model.

When the first detection object is the second vehicle, the number of the ground contact lines of the second vehicle is related to the number of sides of the second vehicle shown in the image to be detected.

As shown in fig. 8a, the first detection object is a vehicle located in the right front of the first vehicle in the image to be detected, and in the case where the left side surface and the rear side surface of the second vehicle are shown in the image to be detected, the ground contact line of the second vehicle includes two ground contact lines, which are the ground contact line of the left side surface of the second vehicle and the ground contact line of the rear side surface of the second vehicle, respectively.

The ground contact line of the left side surface of the second vehicle is as follows: a line between a touchdown point of a tire on the left front side of the second vehicle and a touchdown point of a tire on the left rear side of the second vehicle.

The ground contact line of the rear side surface of the second vehicle is: a line connecting a contact point of a tire on the left rear side of the second vehicle and a contact point of a tire on the right rear side of the second vehicle.

Alternatively, as shown in fig. 8b, the first detection object is a vehicle located directly in front of the first vehicle in the image to be detected, and in the case where only the rear side of the second vehicle is shown in the image to be detected, the ground contact line of the second vehicle includes one ground contact line, which is the ground contact line of the rear side of the second vehicle.

The ground contact line of the rear side of the second vehicle is: a line connecting a contact point of a tire on the left rear side of the second vehicle and a contact point of a tire on the right rear side of the second vehicle.

S103, the first vehicle determines the three-dimensional information of the first detection object according to the passable area boundary and the contact line.

The three-dimensional information of the first detection object is used to determine at least one of a size, a direction, and a relative position of the first detection object.

And the three-dimensional information of the first detection object is used for representing the three-dimensional information of the first detection object shown in the image to be detected. The first vehicle can convert the three-dimensional information of the first detection object shown in the image to be detected into a three-dimensional coordinate system to further determine the real three-dimensional information of the first detection object.

For example, in the case where the first detection object is a second vehicle, the first vehicle converts three-dimensional information shown by the second vehicle in the image to be detected into a three-dimensional coordinate system, and the size of the second vehicle (for example, the length and width of the second vehicle), the direction of the second vehicle (for example, the heading of the second vehicle, the possible traveling direction of the second vehicle), and the position of the second vehicle in the three-dimensional coordinate system can be determined.

It is to be noted that the relative position of the first detection object is related to a three-dimensional coordinate system to which the first vehicle converts the first detection object. For example, when the first vehicle converts the first detection object to the body coordinate system of the first vehicle, the relative position of the first detection object is the position of the first detection object with respect to the first vehicle; when the first vehicle converts the first detection object to the world coordinate system, the relative position of the first detection object is the actual geographic position of the first detection object.

In one possible implementation, the three-dimensional information of the first detection object includes a plurality of points and a plurality of line segments. The plurality of points are projections of the end point of the first detection object displayed in the image to be detected on the ground. The line segments at least comprise the line segments generated by projection of the outermost periphery boundary of the first detection object on the ground; or, the plurality of line segments are contour lines of the first detection object in the image to be detected. The first vehicle inputs the plurality of line segments into a vehicle body coordinate system of the first vehicle, and at least one of the size, direction, and relative position of the first detection object can be determined.

Referring to fig. 7, as shown in fig. 9, S102 may be implemented by S1021 to S1023. S1021-S1023 are described in detail below.

S1021, inputting the image to be detected into the neural network model by the first vehicle to obtain L points.

Among them, the image to be detected usually includes one or more detection objects. The L points are points on the passable region boundary of the one or more detection objects. L is a positive integer.

In a possible implementation manner, the neural network model is a pre-trained neural network model. The neural network model has the capability of marking the borders of the passable area of the detected objects in the image to be detected. The border of the passable area is the border of the detection object in the image to be detected.

Specifically, after acquiring an image to be detected, a first vehicle calls a neural network model, inputs the image to be detected into the neural network model, and outputs L points. The L points are the capability of detecting the passable area boundary of the object in the image to be detected.

Each of the L points output by the neural network model may correspond to an identity. The identification is used to characterize on which side of the detection object the point is located.

For example, a point located on the left side of the detection object corresponds to the first identifier. Accordingly, the first identifier is used to characterize that the point is located on the left side of the detection object.

And the point positioned on the right side of the detection object corresponds to the second identifier. Accordingly, the second identifier is used to characterize that the point is located on the right side of the detection object.

And the point positioned at the front side of the detection object corresponds to the third identifier. Accordingly, the third mark is used to characterize that the point is located on the front side of the detection object.

And the point positioned at the rear side of the detection object corresponds to the fourth identifier. Accordingly, the fourth mark is used to characterize that the point is located at the rear side of the detection object.

S1022, the first vehicle specifies M points from the L points.

The M points are points located on the passable region boundary of the first detection object.

In a specific implementation manner, the first vehicle classifies the L points according to one or more detection objects in the image to be detected, and determines the detection object corresponding to each point. After that, the first vehicle determines M points corresponding to the first detection object from the detection object of each point object. The M points are points located on the border of the passable region of the first detection object. M is a positive integer, and M is less than or equal to L.

S1023, fitting the M points by the first vehicle, and determining a contact line of the first detection object.

In a possible implementation manner, the first vehicle may adopt a random sample consensus (RANSAC) fitting algorithm to fit and determine the touchline of the target object.

Specifically, the first vehicle may employ a RANSAC fitting algorithm, and the process of fitting to determine the touchdown of the target object includes the following steps a-f, as described in detail below:

step a, the first vehicle determines K points which are located at the border of the passable area of the first detection object and have the same identification, wherein K is a positive integer.

And b, randomly selecting T points from the K points by the first vehicle, and fitting the T points by using a least square method to obtain a straight line.

And c, the first vehicle determines the distance from each point except the T point in the K points to the straight line.

And d, determining the points with the distance smaller than the first threshold value as inner group points by the first vehicle, and determining the number of the inner group points.

And e, the first vehicle executes the steps b to d for multiple times, and the multiple straight lines and the number of the inner group points corresponding to each straight line in the multiple straight lines are determined.

And f, the first vehicle determines the straight line with the largest number of corresponding inner group points in the plurality of straight lines as a contact line of the first detection object.

It should be noted that, in the above step e, the greater the number of straight lines determined by the first vehicle through steps b to d, the higher the accuracy of the final determination result.

Based on the technical scheme, the first vehicle can determine the passable area boundary of the first detection object and the contact line of the first detection object by adopting a neural network model and a corresponding fitting algorithm according to the image to be detected.

The monocular camera of the first vehicle can capture an image of an object located in front of the monocular camera. When the second vehicle is in a position directly in front of the monocular camera, the second vehicle typically has only one face that can be captured by the monocular camera of the first vehicle. When the second vehicle is located in front of the monocular camera and deviates from the position right in front of the monocular camera, the second vehicle generally has two surfaces which can be collected by the monocular camera of the first vehicle.

Thus, the image of the second vehicle acquired by the first vehicle includes the following two scenarios: scene 1, the first vehicle acquires two sides of the second vehicle. Scene 2, the first vehicle acquires one side of the second vehicle. The following describes the above scenarios 1 and 2 in detail:

scene 1, the first vehicle acquires two sides of the second vehicle.

The image information of the second vehicle acquired by the first vehicle is related to the image of which direction of the first vehicle acquired by the monocular camera.

For example, when a monocular camera captures an image in front of a first vehicle:

if the first vehicle of the second vehicle runs in the same direction and is positioned in the left front of the first vehicle, the monocular camera can acquire the right side and the rear side of the second vehicle.

If the second vehicle runs in the same direction as the first vehicle and is positioned at the right front of the first vehicle, the monocular camera can acquire the left side surface and the rear side surface of the second vehicle.

If the second vehicle and the first vehicle run in opposite directions and are positioned in the left front of the vehicle, the monocular camera can acquire the front side and the left side of the second vehicle.

If the second vehicle and the first vehicle run in opposite directions and are located in the front right of the vehicle, the monocular camera can acquire the front side face and the right side face of the second vehicle.

For another example, when the monocular camera captures an image of the left side of the second vehicle:

if the second vehicle is located on the left side of the first vehicle and is offset from the position right in front of the monocular camera, the monocular camera may capture the right side of the second vehicle and one of the front side or the rear side of the second vehicle.

In addition, the monocular camera may also acquire images of other side surfaces of the second vehicle, which is not described herein again.

Scene 2, the first vehicle acquires one side of the second vehicle.

if the second vehicle runs in the same direction as the first vehicle and is positioned right in front of the first vehicle, the monocular camera can acquire the rear side face of the second vehicle.

If the second vehicle and the first vehicle run in opposite directions and are located right in front of the first vehicle, the monocular camera can acquire the front side face of the second vehicle.

If the first vehicle runs towards the north and the second vehicle runs towards the east, the monocular camera can acquire the right side of the second vehicle.

If the first vehicle runs towards the north and the second vehicle runs towards the west, the monocular camera can acquire the left side of the second vehicle.

if the second vehicle is located on the left side of the first vehicle and is located right ahead of the monocular camera, the monocular camera can acquire the right side face of the second vehicle.

In the above, it is stated that under different scenarios, the number of the side faces of the second vehicle that can be collected by the first vehicle is different.

It should be noted that, when the number of the side surfaces of the second vehicle collected by the first vehicle is different, the first vehicle determines that the boundary of the passable area of the second vehicle is different, the first vehicle determines that the number of the ground contact lines of the second vehicle is different, and the first vehicle determines that the three-dimensional information of the second vehicle is different.

Specifically, in a scenario where a first vehicle acquires both sides of a second vehicle: the two lateral passable area boundaries are determined by the first vehicle and are used as passable area boundaries of the second vehicle. The first vehicle defines a touchdown line for the second vehicle that includes a first touchdown line and a second touchdown line, the first touchdown line and the second touchdown line respectively corresponding to different ones of the two sides. And the three-dimensional information of the second vehicle determined by the first vehicle comprises the three-dimensional information formed by the two side surfaces.

In a scenario where a first vehicle acquires one side of a second vehicle: the passable area boundary of the second vehicle determined by the first vehicle is the one lateral passable area boundary. The first vehicle defines a touchdown line for the second vehicle that includes a third touchdown line, the third touchdown line being the one side touchdown line. And the three-dimensional information of the second vehicle determined by the first vehicle comprises the three-dimensional information of the side surface.

Therefore, with the above-mentioned scene 1 and scene 2, in S103, the first vehicle determines the three-dimensional information of the first detection object according to the passable area boundary and the touchable line, which includes the following two cases, respectively: the method comprises the following steps that 1, a first vehicle determines three-dimensional information of a first detection object according to the passable area boundary of two side surfaces of the first detection object and a touchdown line; and case 2, the first vehicle determines the three-dimensional information of the first detection object according to the passable area boundary of one side surface of the first detection object and the touchdown line.

Hereinafter, case 1 and case 2 will be described in detail, respectively:

in case 1, the first vehicle determines three-dimensional information of the first detection object based on the passable region boundary of both side surfaces of the first detection object and the touchdown line.

With reference to S102, in case 1, the passable area boundary of the first detection object determined by the first vehicle according to S102 and the touchdown line of the first detection object respectively have the following characteristics:

1. the passable area boundary of the first detection object comprises a plurality of boundary points corresponding to the first identifier and a plurality of boundary points corresponding to the second identifiers.

The first mark also corresponds to a first side of the first detection object, and the second mark also corresponds to a second side of the first detection object. The first side face and the second side face are two intersecting side faces in the first detection object.

2. The contact ground wire comprises a first contact ground wire and a second contact ground wire.

The first touchdown line is a touchdown line determined by fitting a plurality of boundary points corresponding to the first identifier.

The second touchdown line is a touchdown line determined by fitting a plurality of boundary points corresponding to the second identifier.

3. The plurality of boundary points corresponding to the first identifier comprise first boundary points; the first boundary point is the boundary point with the largest distance with the second contact line in the plurality of boundary points with the first marks.

With reference to S103, in case 1, the first vehicle is determined according to the three-dimensional information of the first detection object determined in S103, and according to three points and two lines corresponding to the first detection object.

Wherein a first point of the three points is a projection of the first boundary point on the ground.

The second point of the three points is a projection of the second boundary point on the ground.

The third point of the three points is a second point, and an intersection of a straight line parallel to the second ground contact line and the first ground contact line.

The first of the two lines is the line between the first point and the third point.

The second line of the two lines is the line between the second point and the third point.

Referring to fig. 7, as shown in fig. 10, in case 1, S103 may be specifically implemented by the following S1031 to S1035. S1031 to S1035 are specifically described below:

and S1031, determining a first point by the first vehicle according to the first boundary point and the first contact line.

Wherein the first point is an intersection of the first contact line and the first straight line. The first straight line is a straight line which is perpendicular to the visual flat line in the image to be detected and is the first boundary point.

In a specific implementation manner, in combination with the first detection object in the image to be detected shown in fig. 11, the method for the first vehicle to determine the first point is as follows:

step I, a first vehicle determines a first straight line; the first straight line is a straight line which is perpendicular to the visual flat line and is a first boundary point in the image to be detected.

In one implementation, the first vehicle crosses the first boundary point as a perpendicular to the eye level, the perpendicular being the first line.

And step II, the first vehicle determines that the intersection point of the first straight line and the first contact line is a first point.

In one implementation, the first vehicle is an extension of the first ground contact line, and the extension intersects the first straight line and is at a point a. The first vehicle determines the point a as a first point.

S1032, the first vehicle determines a second point according to the first contact line, the second contact line and the second boundary point.

Wherein the second point is an intersection of the second line and the third line. The second straight line is a straight line of an intersection point of the first contact ground line and the view flat line and a vertex of the second contact ground line far away from the first contact ground line. The third straight line is a straight line which is perpendicular to the visual flat line in the image to be detected and is the second boundary point.

In a specific implementation manner, in combination with the first detection object in the image to be detected shown in fig. 11, the method for the first vehicle to determine the second point is as follows:

and step III, determining a second straight line by the first vehicle.

The second straight line is a straight line which is positioned at the intersection point of the first ground contact line and the visual flat line in the image to be detected and is far away from the endpoint of the first ground contact line in the second ground contact line.

In one implementation, the first vehicle is used as an extension line of the grounding wire to obtain an intersection point b of the first grounding wire and the horizon line. The first vehicle determines an end point c of the second contact line that is far from the first contact line. The first vehicle makes a straight line passing through the intersection point b and the end point c, and the straight line is a second straight line.

And IV, determining a third straight line by the first vehicle.

The third straight line is a straight line which is perpendicular to the visual flat line and is a second boundary point in the image to be detected.

In one implementation, the first vehicle crosses the second boundary point to make a perpendicular to the eye level, and the perpendicular is a third straight line.

And V, the first vehicle determines that the intersection point of the second straight line and the third straight line is a second point.

In one implementation, the first vehicle determines that the second line and the third line intersect at a point c, and the first vehicle determines that the point c is the second point.

And S1033, determining a third point by the first vehicle according to the first contact line, the second contact line and the second point.

The third point is a first contact line and an intersection point of the first contact line and the fourth straight line. The fourth straight line is a straight line which passes through the second point in the image to be detected and is parallel to the second ground contact line.

In a specific implementation manner, in combination with the first detection object in the image to be detected shown in fig. 11, the method for the first vehicle to determine the third point is as follows:

and VI, determining a fourth straight line by the first vehicle.

In one implementation, the first vehicle makes a parallel line to the second ground contact line at a second point. The first vehicle determines the parallel line as a fourth straight line.

And step VII, the first vehicle determines that the intersection point of the fourth straight line and the first contact line is a third point.

In one implementation, the determination device determines an intersection d of the fourth straight line and the first ground contact line, and the first vehicle determines the intersection d as the third point.

S1034, the first vehicle determines a first line according to the first point and the third point.

In one implementation, as shown in fig. 11, the first vehicle makes a line segment a with the first point and the third point as end points, and the first vehicle determines the line segment as a first line.

And S1035, determining a second line by the first vehicle according to the second point and the third point.

In one implementation, as shown in fig. 11, the first vehicle makes a line segment b with the second point and the third point as end points, and the first vehicle determines the line segment as the second line.

And 2, determining the three-dimensional information of the first detection object by the first vehicle according to the passable area boundary of one side surface of the first detection object and the touchdown line.

With reference to S102, in case 2, the passable area boundary of the first detection object determined by the first vehicle according to S102 and the touchdown line of the first detection object respectively have the following characteristics:

a. the border of the passable area of the first detection object comprises a plurality of border points corresponding to the third identifier; the third mark also corresponds to a third side of the first detection object.

b. The ground contact line of the first detection object comprises a third ground contact line; the third touchdown line is determined by fitting a plurality of boundary points corresponding to the third identifier.

c. The boundary points with the third identification comprise a third boundary point and a fourth boundary point.

The third boundary point is a point farthest from one end of the third contact line among the plurality of boundary points having the third mark.

The fourth boundary point is the farthest point from the other end of the third contact line among the plurality of boundary points having the third identification.

With reference to S103, in case 2, the first vehicle is determined according to the three-dimensional information of the first detection object determined in S103, and according to two points and one line corresponding to the first detection object.

The first of the two points is a projection of the third boundary point on the ground.

The second of the two points is a projection of the fourth boundary point on the ground.

Referring to fig. 8, as shown in fig. 12, in case 2, S103 may be implemented by following S1036 to S1038, and S1036 to S1038 is described in detail below.

S1036, the first vehicle determines a first point according to the third boundary point and the third contact line.

Wherein the first point is an intersection of the third ground contact line and the fifth straight line. The fifth straight line is a straight line which is perpendicular to the third touchdown line and is a third boundary point.

In a specific implementation manner, in combination with the first detection object in the image to be detected shown in fig. 13, the method for the first vehicle to determine the first point of the three-dimensional information of the first detection object is as follows:

step 1, the first vehicle determines a fifth straight line.

And the fifth straight line is a straight line which is perpendicular to the visual flat line and is a third boundary point in the image to be detected.

In one implementation, the first vehicle makes a perpendicular to the eye level line through the third boundary point, and the first vehicle determines that the perpendicular is a fifth straight line.

And 2, determining the intersection point of the fifth straight line and the third contact line as a first point by the first vehicle.

In one implementation, the first vehicle is used as an extension line of the third touch ground line, the extension line of the third touch ground line intersects with the fifth straight line at a point e, and the first vehicle determines that the point e is the first point.

And S1037, determining a second point by the first vehicle according to the fourth boundary point and the third contact line.

And the second point is the intersection point of the third contact line and the sixth straight line. The sixth straight line is a straight line perpendicular to the third ground contact line and is a fourth boundary point.

In a specific implementation manner, in combination with the first detection object in the image to be detected shown in fig. 13, the method for the first vehicle to determine the second point of the three-dimensional information of the first detection object is as follows:

and 3, determining a sixth straight line by the first vehicle.

And the sixth straight line is a straight line which is perpendicular to the visual flat line and is a fourth boundary point in the image to be detected.

In one implementation, the first vehicle makes a perpendicular to the eye level at the fourth boundary point, and the first vehicle determines that the perpendicular is a sixth straight line.

And 4, determining the intersection point of the sixth straight line and the third contact line as a second point by the first vehicle.

In one implementation, the first vehicle is used as an extension line of the third touch ground line, the extension line of the third touch ground line intersects with the sixth straight line at a point f, and the first vehicle determines that the point f is the second point.

And S1038, the first vehicle determines a first line according to the first point and the second point.

In one implementation, as shown in fig. 13, the first vehicle makes a line segment c with the first point and the second point as end points, and the first vehicle determines that the line segment c is the first line.

Based on the technical scheme, the first vehicle can determine the three-dimensional information of the first detection object according to the passable area boundary of the first detection object and the touchdown line of the first detection object.

It should be noted that the three-dimensional information is three-dimensional information represented by the first detection object in the image to be detected. When the first vehicle needs to determine the real three-dimensional information of the first detection object, the three-dimensional information represented by the first detection object in the image to be detected needs to be brought into a vehicle body coordinate system with the first vehicle, so that the first vehicle determines information such as the position of the first detection object relative to the first vehicle, the size (at least one of length, width and height) of the first detection object, and the orientation of the first detection object.

Specifically, referring to fig. 7, as shown in fig. 9, after S103, the method further includes:

s104, the first vehicle inputs the three-dimensional information of the first detection object into a vehicle body coordinate system, and at least one of the size, the direction and the relative position of the first detection object is determined.

In a specific implementation manner, a first rectangular coordinate system is established by a first vehicle according to an image to be detected; the image to be detected is located in the first rectangular coordinate system. The first rectangular coordinate system may be a matrix set for the image capturing device in advance, and the pictures captured by the image capturing device may be mapped to the matrix.

The first vehicle determines coordinates in an inter-first coordinate system of the three-dimensional information of the first detection object. After this, the first vehicle determines the intrinsic and extrinsic parameters of the image acquisition device. And the first vehicle converts the coordinates of the three-dimensional information of the first detection object in the first rectangular coordinate system into the coordinates in the vehicle body coordinate system according to the internal parameters and the external parameters of the image acquisition device and the position of the image acquisition device in the vehicle body coordinate system.

And the first vehicle determines the information such as the position, the movement direction and the size of the first detection object according to the coordinates of the three-dimensional information of the first detection object in the vehicle body coordinate system.

The internal parameters of the image acquisition device are used for representing some parameters related to the image acquisition device, such as the focal length of the image acquisition device, the pixel size and the like.

And the external parameters of the image acquisition device are used for representing the parameters of the image acquisition device in a world coordinate system, such as the position, the rotation direction and the like of the image acquisition device in the world coordinate system.

It should be noted that the first vehicle is provided with an internal reference matrix and an external reference matrix of the image capturing device in advance. When the first vehicle converts one coordinate point in the body coordinate system into one coordinate point in the inter-first coordinate system (denoted as a first coordinate point): and the first vehicle multiplies the first coordinate point by the external reference matrix and the internal reference matrix in sequence to obtain a corresponding coordinate point of the first coordinate point in the first rectangular coordinate system. When the first vehicle finds the corresponding point in the vehicle body coordinate system according to the coordinate point in the first rectangular coordinate system, the corresponding point in the vehicle body coordinate system can be determined according to the coordinate point in the first rectangular coordinate system only by executing the operation process opposite to the above operation process.

Note that after S105, the first vehicle determines the size of the first detection object, including the length and width of the first monitoring object.

To determine the height of the first detection object, the first vehicle may determine the height of the first monitoring object according to the type of the first detection object.

For example, in the case where the first detection object is the second vehicle, the first vehicle identifies the vehicle type of the second vehicle, and determines the height of the vehicle according to the vehicle type.

An example, the vehicle type of the second vehicle is a class of vehicles, such as: a mini-car, a small-car, a compact-car, a medium-large car, a large-car, a small Sport Utility Vehicle (SUV), a compact SUV, a medium-sized SUV, a large-sized SUV, a compact multi-purpose vehicle (MPV), a medium-sized MPV, a large-sized MPV, a sports car, a pickup, a micro-deck, a light bus, a micro truck, and the like.

The first vehicle is pre-configured with standard sizes of vehicles of various grades, and after the first vehicle determines the vehicle grade of the second vehicle, the height of the second vehicle is determined according to the vehicle grade of the second vehicle.

As yet another example, the vehicle type of the second vehicle is a model of the vehicle (e.g., vehicle make + specific vehicle type). The first vehicle is pre-configured with standard sizes of vehicles of various types, and after the first vehicle determines the vehicle type of the second vehicle, the height of the second vehicle is determined according to the vehicle type of the second vehicle.

It is noted that according to the above method, the first vehicle may also determine the length and width of the second vehicle according to the type of the second vehicle. The first vehicle can determine the exact length and width of the second vehicle based on the length and width of the second vehicle determined by the method and the length and width determined from the three-dimensional information of the second vehicle in the image to be detected.

In one possible implementation, after S105, the first vehicle determines the positions, sizes, and moving directions of all vehicles in the image to be detected. The first vehicle plans a driving route of the first vehicle according to the positions, sizes and moving directions of all vehicles, current road information, obstacle information, destination information of the vehicles and the like determined by other devices in the first vehicle.

After the first vehicle determines the driving route of the first vehicle, a first control instruction is generated according to the first driving route. The first control instruction is used for instructing the first vehicle to travel according to the planned travel route.

And the first vehicle issues the first control instruction to the first vehicle. The first vehicle carries out intelligent driving according to a control command issued by the first vehicle.

Based on the technical scheme, the first vehicle can determine the size and the direction of the second vehicle and the position of the second vehicle relative to the first vehicle according to the three-dimensional information of the second vehicle. After that, the first vehicle can plan the driving route of the first vehicle according to the information, so as to realize intelligent driving.

All the schemes in the above embodiments of the present application can be combined without contradiction.

The above description mainly introduces the solution of the embodiment of the present application from the perspective of interaction between the respective devices. It is to be understood that each device, for example, the first vehicle and the second vehicle, includes at least one of a hardware structure and a software module corresponding to each function in order to realize the functions. Those of skill in the art would readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is understood that, in order to realize the functions in the above-described embodiments, the vehicle includes a corresponding hardware structure and/or software module that performs each function. Those of skill in the art will readily appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software driven hardware depends on the particular application scenario and design constraints imposed on the solution.

Fig. 14 is a schematic structural diagram of an apparatus for determining three-dimensional information of a detection object according to an embodiment of the present application. The device for determining the three-dimensional information of the detection object can be used for realizing the functions of the processor in the method embodiment, so that the beneficial effects of the method embodiment can be realized. In an embodiment of the present application, the means for determining three-dimensional information of the detection object may be a processor 161 as shown in fig. 1.

As shown in fig. 14, an apparatus 1400 for determining three-dimensional information of a detection object includes a processing unit 1410 and a communication unit 1420. The apparatus 1400 for determining three-dimensional information of a detection object is used to implement the function of the first vehicle in the method embodiment shown in fig. 7, 9, 10, or 12 described above.

When the apparatus 1400 for determining three-dimensional information of a detection object is used to implement the function of a processor in the method embodiment shown in fig. 7: the processing unit 1410 is configured to execute S102 to S103, and the communication unit 1420 is configured to communicate with other entities.

When the apparatus 1400 for determining three-dimensional information of a detection object is used to implement the function of a processor in the method embodiment shown in fig. 9: the processing unit 1410 is configured to perform S101, S1021 to S1023, S103 and S104, and the communication unit 1420 is configured to communicate with other entities.

When the apparatus 1400 for determining three-dimensional information of a detection object is used to implement the function of a processor in the method embodiment shown in fig. 10: the processing unit 1410 is configured to execute S101, S102, and S1031 to S1035, and the communication unit 1420 is configured to communicate with other entities.

When the apparatus 1400 for determining three-dimensional information of a detection object is used to implement the function of a processor in the method embodiment shown in fig. 12: the processing unit 1410 is configured to perform S101, S102, and S1036 to S1038, and the communication unit 1420 is configured to communicate with other entities.

More detailed descriptions about the processing unit 1410 and the communication unit 1420 can be directly obtained by referring to fig. 7 and fig. 9, and related descriptions in the method embodiments shown in fig. 10 or fig. 12, which are not repeated herein.

In implementation, the steps of the method provided by this embodiment may be implemented by hardware integrated logic circuits in a processor or instructions in the form of software. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

Processors in the present application may include, but are not limited to, at least one of: various computing devices that run software, such as a Central Processing Unit (CPU), a microprocessor, a Digital Signal Processor (DSP), a Microcontroller (MCU), or an artificial intelligence processor, may each include one or more cores for executing software instructions to perform operations or processing. The processor may be a single semiconductor chip or integrated with other circuits to form a semiconductor chip, for example, an SoC (system on chip) with other circuits (such as a codec circuit, a hardware acceleration circuit, or various buses and interface circuits), or may be integrated in the ASIC as a built-in processor of the ASIC, which may be packaged separately or together with other circuits. The processor may further include necessary hardware accelerators such as Field Programmable Gate Arrays (FPGAs), PLDs (programmable logic devices), or logic circuits implementing dedicated logic operations, in addition to cores for executing software instructions to perform operations or processes.

The memory in the embodiment of the present application may include at least one of the following types: read-only memory (ROM) or other types of static memory devices that may store static information and instructions, Random Access Memory (RAM) or other types of dynamic memory devices that may store information and instructions, and Electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory may also be, but is not limited to, a compact disk-read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

Embodiments of the present application also provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to perform any of the above methods.

Embodiments of the present application also provide a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the methods described above.

An embodiment of the present application further provides a communication system, including: the base station and the server.

Embodiments of the present application further provide a chip, where the chip includes a processor and an interface circuit, where the interface circuit is coupled to the processor, the processor is configured to execute a computer program or instructions to implement the method, and the interface circuit is configured to communicate with other modules outside the chip.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented using a software program, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the present application are all or partially generated upon loading and execution of computer program instructions on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). Computer-readable storage media can be any available media that can be accessed by a computer or can comprise one or more data storage devices, such as servers, data centers, and the like, that can be integrated with the media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

While the present application has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Although the present application has been described in conjunction with specific features and embodiments thereof, it will be evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and figures are merely exemplary of the present application as defined in the appended claims and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Finally, it should be noted that: the above description is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of determining three-dimensional information of a test object, comprising:

acquiring an image to be detected; the image to be detected comprises a first detection object;

determining a passable area boundary of the first detection object and a touchdown line of the first detection object; the passable area boundary comprises a boundary of the first detection object in the image to be detected; the ground contact line is a connecting line of an intersection point of the first detection object and the ground;

and determining the three-dimensional information of the first detection object according to the border of the passable area and the grounding wire.

2. The method of claim 1, wherein the three-dimensional information of the first detected object is used to determine at least one of a size, a direction, and a relative position of the first detected object.

3. The method according to claim 1 or 2, wherein the passable area boundary of the first detection object comprises a plurality of boundary points corresponding to the first identifier and a plurality of boundary points corresponding to the second identifier; the first identification also corresponds to a first side face of the first detection object, and the second identification also corresponds to a second side face of the first detection object; the first side surface and the second side surface are two intersected side surfaces in the first detection object;

the contact ground wire comprises a first contact ground wire and a second contact ground wire;

the first ground contact line is a ground contact line determined by fitting the plurality of boundary points corresponding to the first identifier;

the second touchdown line is a touchdown line determined by fitting the plurality of boundary points corresponding to the second identifier.

4. The method of claim 3, wherein the plurality of boundary points corresponding to the first identifier comprises a first boundary point; the first boundary point is the boundary point with the largest distance with the second contact line in a plurality of boundary points with first marks;

the plurality of boundary points corresponding to the second identifier comprise a second boundary point; the second boundary point is a boundary point with the largest distance from the first contact line among the plurality of boundary points with the second identifier.

5. The method of claim 4, wherein the determining three-dimensional information of the first detected object comprises:

determining a projection of the first boundary point on the ground as a first point;

determining a projection of the second boundary point on the ground as a second point;

determining the projection of the second boundary point on the ground, wherein the intersection point of a straight line parallel to the second ground contact line and the first ground contact line is a third point;

determining a connection line between the first point and the third point as a first line;

determining a connection line between the second point and the third point as a second line;

determining three-dimensional information of the first detection object from the first point, the second point, the third point, the first line, and the second line.

6. The method according to claim 1 or 2, wherein the passable area boundary of the first detection object comprises a plurality of boundary points corresponding to a third marker; the third mark also corresponds to a third side of the first detection object; the ground contact line of the first detection object comprises a third ground contact line; the third touchdown line is determined by fitting the plurality of boundary points corresponding to the third identifier.

7. The method of claim 6, wherein the plurality of boundary points with the third identification comprises a third boundary point and a fourth boundary point;

the third boundary point is the point which is farthest from one end of the third contact line in the plurality of boundary points with the third identification;

the fourth boundary point is a point farthest from the other end of the third contact line among the plurality of boundary points having the third identifier.

8. The method of claim 7, wherein the determining three-dimensional information of the first detected object comprises:

determining a projection of the third boundary point on the ground as a first point;

determining the projection of the fourth boundary point on the ground as a second point;

determining a connection line between the first point and the second point as a first line;

and determining three-dimensional information of the first detection object according to the first point, the second point and the first line.

9. The method according to any one of claims 1-8, further comprising:

inputting the three-dimensional information of the first detection object into a vehicle body coordinate system, and determining at least one of the size, the direction and the relative position of the first detection object.

10. An apparatus for determining three-dimensional information of an object under inspection, comprising: a communication unit and a processing unit;

the communication unit is used for acquiring an image to be detected; the image to be detected comprises a first detection object;

the processing unit is used for determining the border of the passable area of the first detection object and the touchdown line of the first detection object; the passable area boundary comprises a boundary of the first detection object in the image to be detected; the ground contact line is a connecting line of an intersection point of the first detection object and the ground;

the processing unit is further configured to determine three-dimensional information of the first detection object according to the passable area boundary and the grounding line.

11. An apparatus for determining three-dimensional information of a test object, the apparatus comprising a processor and a memory, wherein the memory is configured to store a computer program and instructions, and the processor is configured to execute the computer program and instructions to implement the method of determining three-dimensional information of a test object according to any one of claims 1-9.

12. An intelligent vehicle, characterized by comprising a vehicle body, a monocular camera and the device for determining three-dimensional information of a detection object as claimed in claim 10, wherein the monocular camera is used for collecting an image to be detected; the apparatus for determining three-dimensional information of an object to be detected is used for executing the method for determining three-dimensional information of an object to be detected according to any one of claims 1 to 9, and determining the three-dimensional information of the object to be detected.

13. The smart vehicle of claim 12, further comprising a display screen; the display screen is used for displaying the three-dimensional information of the detection object.

14. Advanced driving assistance system, ADAS, comprising means for determining three-dimensional information of a detection object according to claim 10, for performing a method for determining three-dimensional information of a detection object according to any of claims 1-9, determining three-dimensional information of a detection object.

15. A computer-readable storage medium comprising computer instructions which, when executed on a computer, cause the computer to perform the method of determining three-dimensional information of a detection object according to any one of claims 1-9.

16. A computer program product, which, when run on a computer, causes the computer to perform the method of determining three-dimensional information of a test object according to any one of claims 1 to 9.