CN116163618A

CN116163618A - Vehicle door control method, vehicle door control device, vehicle-mounted terminal, vehicle, and medium

Info

Publication number: CN116163618A
Application number: CN202111413377.6A
Authority: CN
Inventors: 范佳
Original assignee: BYD Co Ltd
Current assignee: BYD Co Ltd
Priority date: 2021-11-25
Filing date: 2021-11-25
Publication date: 2023-05-26

Abstract

The invention discloses a vehicle door control system and a vehicle door control method. The vehicle door control system comprises a vehicle-mounted camera, a vehicle-mounted radar and a controller, wherein the vehicle-mounted camera is used for collecting image information of obstacles around a vehicle, the vehicle-mounted radar is used for collecting ultrasonic information of the obstacles around the vehicle, the controller is respectively connected with the vehicle-mounted camera and the vehicle-mounted radar, the controller is used for carrying out fusion processing on the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, obtaining a space state change relation between the obstacles around the vehicle and the vehicle door, and controlling the vehicle door according to the space state change relation. According to the vehicle door control method, the vehicle door control device, the vehicle-mounted terminal, the vehicle and the medium, the intelligent degree of the vehicle door can be improved, and the safety and the reliability of the vehicle door are improved.

Description

Vehicle door control method, vehicle door control device, vehicle-mounted terminal, vehicle, and medium

Technical Field

The present invention relates to the field of vehicle doors, and in particular, to a door control method, a door control device, a vehicle-mounted terminal, a vehicle, and a medium.

Background

With the rapid development of science and technology, vehicle intellectualization has become a trend. In the related art, regarding the scheme of door intellectualization, only the implementation of the door lock system is focused, and the influence of environmental factors on the door opening and closing process is ignored. For example, in the intelligent scheme of the flat-open type car door, only the automatic door opening and closing are realized, and the situation that environmental factors cause obstruction to the car door opening and closing process is not considered, so that the intelligent degree of the car door is not high, and the user experience is affected.

Disclosure of Invention

The present invention aims to solve at least one of the technical problems in the related art to some extent. Therefore, an object of the present invention is to provide a door control system that can improve the degree of intelligence of a door and improve the safety and reliability of the door.

A second object of the present invention is to provide a vehicle door control method.

In order to achieve the above object, an embodiment of a first aspect of the present invention provides a vehicle door control system, which includes a vehicle-mounted camera, a vehicle-mounted radar, and a controller, wherein the vehicle-mounted camera is used for collecting image information of obstacles around a vehicle, the vehicle-mounted radar is used for collecting ultrasonic information of the obstacles around the vehicle, the controller is respectively connected with the vehicle-mounted camera and the vehicle-mounted radar, and the controller is used for performing fusion processing on the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle to obtain a spatial state change relationship between the obstacles around the vehicle and the vehicle door, and controlling the vehicle door according to the spatial state change relationship.

According to the vehicle door control system provided by the embodiment of the invention, the vehicle door is controlled according to the spatial state change relation of the surrounding obstacles of the vehicle relative to the vehicle door, so that the intelligent degree of the vehicle door is improved, and the safety and reliability of the vehicle door are improved.

In some embodiments of the present invention, the controller is further configured to establish a spatial rectangular coordinate system according to a positional relationship between the vehicle-mounted camera and the vehicle-mounted radar, determine three-dimensional coordinates of the vehicle-surrounding obstacle in the spatial rectangular coordinate system according to image information of the vehicle-surrounding obstacle and ultrasonic information of the vehicle-surrounding obstacle, and determine a spatial state change relationship between the vehicle-surrounding obstacle and the vehicle door according to the three-dimensional coordinates of the vehicle-surrounding obstacle in the spatial rectangular coordinate system.

In some embodiments of the present invention, the controller is further configured to determine three-dimensional coordinates of the vehicle door in the space rectangular coordinate system, calculate a distance between the three-dimensional coordinates of the obstacle around the vehicle in the space rectangular coordinate system and the three-dimensional coordinates of the vehicle door in the space rectangular coordinate system, and determine a spatial state change relationship between the obstacle around the vehicle and the vehicle door according to the distance.

In some embodiments of the invention, the controller is further configured to determine that the obstacle around the vehicle is in a hazardous space of the door when the distance is less than a first threshold; determining that the obstacle around the vehicle is in a safe space of the vehicle door when the distance is greater than a second threshold value; and when the distance is larger than or equal to a first threshold value and smaller than or equal to a second threshold value, determining that the obstacle around the vehicle is in the critical space of the vehicle door, wherein the first threshold value is smaller than the second threshold value.

In some embodiments of the present invention, the controller is further configured to control the door to prohibit execution of the received door opening/closing command when the obstacle around the vehicle is in a dangerous space of the door and the door is in a stopped state; when the obstacle around the vehicle is in the dangerous space of the vehicle door and the vehicle door is in a moving state, controlling the vehicle door to stop moving and performing anti-collision protection; controlling the vehicle door to execute the received door opening and closing command when the obstacle around the vehicle is in a safe space of the vehicle door and the vehicle door is in the stopped state; when the obstacle around the vehicle is in the safety space of the vehicle door and the vehicle door is in the motion state, controlling the vehicle door to execute the received door opening and closing command; when the obstacle around the vehicle is in the critical space of the vehicle door and the vehicle door is in a stop state, controlling the vehicle door to delay executing the received door opening and closing instruction and performing anti-collision protection; and when the surrounding obstacle is in the critical space of the vehicle door and the vehicle door is in a moving state, controlling the vehicle door according to the movement time required by the surrounding obstacle to move to the dangerous space of the vehicle door.

In some embodiments of the present invention, the controller is further configured to control the door to stop moving and perform crash protection when the door is executing a door closing command and the movement duration is less than or equal to a remaining closing duration of the door; and when the door is executing the door closing command and the movement time period is longer than the remaining closing time period of the door, controlling the door to continue executing the door closing command.

In some embodiments of the present invention, the vehicle-mounted radar includes a first radar and a second radar, and the vehicle-mounted camera, the first radar and the second radar are located on the same straight line, and the controller is further configured to establish the space rectangular coordinate system with the vehicle-mounted camera as an origin and the straight line as an x-axis.

In some embodiments of the present invention, the three-dimensional coordinates of the obstacle around the vehicle include a first x-axis component, a first y-axis component, and a z-axis component, wherein the controller is further configured to determine a projection length of a line connecting the first radar and the obstacle around the vehicle on the x-axis based on the ultrasonic information acquired by the first radar and the ultrasonic information acquired by the second radar, and determine the first x-axis component based on a distance between the first radar and the vehicle-mounted camera and the projection length, and determine a two-dimensional coordinates of the obstacle around the vehicle in a two-dimensional imaging image based on the image information, and determine the first y-axis component and the z-axis component based on the first x-axis component and the two-dimensional coordinates, and use the first x-axis component, the first y-axis component, and the z-axis component as the three-dimensional coordinates of the obstacle around the vehicle.

In some embodiments of the present invention, the two-dimensional coordinates include a second x-axis component and a second y-axis component, wherein the controller is further configured to determine a proportional relationship between a shooting size of the in-vehicle camera and a size of the two-dimensional imaging image according to the first x-axis component and the second x-axis component, determine the first y-axis component according to the proportional relationship and the second y-axis component, and determine the z-axis component according to the proportional relationship.

In some embodiments of the present invention, the vehicle door control system further comprises a display, and the display is connected to the controller, where the controller is further configured to control the display to dynamically display the spatial state change relationship.

To achieve the above object, an embodiment of a second aspect of the present invention provides a vehicle door control method, including: acquiring image information of obstacles around a vehicle, and acquiring ultrasonic information of the obstacles around the vehicle; performing fusion processing on the image information of the surrounding obstacles of the vehicle and the ultrasonic information of the surrounding obstacles of the vehicle to obtain a space state change relation between the surrounding obstacles of the vehicle and a vehicle door; and controlling the vehicle door according to the space state change relation.

According to the vehicle door control method provided by the embodiment of the invention, the vehicle door is controlled according to the spatial state change relation of the surrounding obstacles of the vehicle relative to the vehicle door, so that the intelligent degree of the vehicle door is improved, and the safety and reliability of the vehicle door are improved.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

FIG. 1 is a block diagram of a door control system according to one embodiment of the invention;

FIG. 2 is a schematic diagram of intelligent obstacle recognition for a vehicle door control system in accordance with one embodiment of the present invention;

FIG. 3 is a schematic illustration of a scenario of a vehicle door control system according to one embodiment of the invention;

FIG. 4 is a schematic illustration of a scenario of a vehicle door control system according to one embodiment of the invention;

FIG. 5 is a schematic view of a scenario of a door control system according to one embodiment of the invention;

FIG. 6 is a schematic illustration of a scenario of a vehicle door control system according to one embodiment of the invention;

FIG. 7 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

FIG. 8 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

FIG. 9 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

FIG. 10 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

FIG. 11 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

FIG. 12 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

FIG. 13 is a flow chart of a method of controlling a vehicle door according to one embodiment of the invention;

fig. 14 is a flow chart of a door control method according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.

Referring to fig. 1, the present invention proposes a door control system 100, where the door control system 100 includes a vehicle-mounted camera 10, a vehicle-mounted radar 20, and a controller 30, the vehicle-mounted camera 10 is used for collecting image information of obstacles around a vehicle, the vehicle-mounted radar 20 is used for collecting ultrasonic information of obstacles around the vehicle, the controller 30 is respectively connected with the vehicle-mounted camera 10 and the vehicle-mounted radar 20, the controller 30 is used for performing fusion processing on the image information of the obstacles around the vehicle and the ultrasonic information of the obstacles around the vehicle, obtaining a spatial state change relationship between the obstacles around the vehicle and the door 40, and controlling the door 40 according to the spatial state change relationship.

According to the door control system 100 of the embodiment of the invention, the door control is performed according to the spatial state change relation of the surrounding obstacle of the vehicle relative to the door 40, so that the degree of intellectualization of the door 40 is improved, and the safety and reliability of the door 40 are improved.

It is understood that, regarding the recognition of the obstacle around the vehicle, in the related art, the obstacle around the vehicle is generally recognized based on only the ultrasonic information of the obstacle around the vehicle collected by the vehicle-mounted radar, or the obstacle around the vehicle is recognized based on only the image information of the obstacle around the vehicle collected by the vehicle-mounted camera, without fusing the data collected by the vehicle-mounted radar and the vehicle-mounted camera and recognizing the obstacle around the vehicle from the fused data, nor employing the recognition of the obstacle around the vehicle in the door control scheme.

Further, in the related art that recognizes the obstacle around the vehicle based only on the ultrasonic information of the obstacle around the vehicle acquired by the vehicle-mounted radar, according to the ultrasonic principle, ultrasonic waves are transmitted by the vehicle-mounted radar, and after the ultrasonic waves strike the obstacle around the vehicle, a part of the ultrasonic waves are reflected back to be detected by the vehicle-mounted radar, and the actual distance between the vehicle and the obstacle around the vehicle is calculated from the detected reflected waves. However, due to the problems of the installation height angle, the ultrasonic divergence angle and the like, the vehicle-mounted radar has a working blind area, and the vehicle-mounted radar cannot accurately identify the characteristics of the size, the outline and the like of the obstacles around the vehicle. In the related art that recognizes the obstacle around the vehicle based only on the image information of the obstacle around the vehicle acquired by the vehicle-mounted camera, the image information is acquired by the vehicle-mounted camera, and the visual analysis is performed. However, due to the problems of the installation height and the angle of the vehicle-mounted camera, a certain error exists between the actual obstacle information around the vehicle and the displayed information after the image information is analyzed, calculated and restored. In addition, the vehicle-mounted camera is too dependent on environmental factors, for example, in the evening or in a dim-light environment or in rainy and snowy weather, the quality of image information acquired by the vehicle-mounted camera is low, and analysis, calculation and restoration are not facilitated. Meanwhile, if the vehicle-mounted camera is polluted or shielded carelessly, the real and reliable image information is more unfavorable to be acquired.

Moreover, the door control system in the related art focuses only on the realization of automatic door opening and automatic door closing, and does not recognize obstacles around the vehicle and perform automatic door opening and automatic door closing according to a spatial state change relationship of the obstacles around the vehicle. That is, the door control system in the related art has problems of low degree of intelligence, low reliability, and the like.

In the door control system 100 of the present embodiment, considering that the vehicle-mounted radar 20 has the advantage of being less affected by the environment in the working process, for example, in the evening or in a dark place of light, the surrounding obstacle of the vehicle may be detected, and considering that the vehicle-mounted camera 10 has the advantage of being able to realize non-blind area detection depending on the installation position, the image information collected by the vehicle-mounted camera 10 and the ultrasonic information collected by the vehicle-mounted radar 20 are fused and analyzed, and the advantages of both sides are combined to compensate the disadvantages of both sides, thereby ensuring the accuracy of identifying the surrounding obstacle of the vehicle, so that the door control system 100 can be implemented, and the relevant scheme for judging the spatial state change relationship between the surrounding obstacle of the vehicle and the door 40 is added in the door control system 100, so that the door control system 100 of the present embodiment is more intelligent, reliable and safe in operation. Meanwhile, the door control system 100 implemented by the present invention is realized based on the in-vehicle camera 10 and the in-vehicle radar 20 already mounted on the vehicle, and can also effectively control the cost.

Specifically, the door 40 may include a side-hung door, a side-by-side door, a side-slip door, and the like. The actuator of the vehicle door 40 may include a door, a lock, etc.

The in-vehicle camera 10 may include a plurality of in-vehicle cameras 10, and each in-vehicle camera 10 may be oriented differently so as to monitor the environment around the vehicle in multiple directions and collect image information of obstacles around the vehicle. The plurality of in-vehicle cameras 10 may be arranged to be distributed around the body of the vehicle or may be arranged to be gathered on the roof of the vehicle. When the plurality of in-vehicle cameras 10 are gathered on the roof of the vehicle, the plurality of cameras may be provided fixedly installed on the roof of the vehicle or may be provided rotatably installed on the roof of the vehicle, and the plurality of in-vehicle cameras 10 may be provided to rotate synchronously or may be provided to rotate individually.

The in-vehicle camera 10 may also include one, and one in-vehicle camera 10 may be configured to be rotatably mounted on top of the vehicle, thereby achieving the effect of monitoring the environment surrounding the vehicle. The vehicle-mounted camera 10 can shoot the surrounding obstacles of the vehicle in the self-facing range according to the preset sampling frequency to acquire the image information of the surrounding obstacles of the vehicle, and can shoot the surrounding obstacles of the vehicle in the self-facing range in real time in a video recording mode to acquire the image information of the surrounding obstacles of the vehicle.

The vehicle-mounted radar 20 may include a plurality of vehicle-mounted radars 20, and the orientation of each vehicle-mounted radar 20 may be different so as to detect the environment around the vehicle in multiple directions and collect ultrasonic information of obstacles around the vehicle.

The obstacle around the vehicle may be a person or object movable around the vehicle, such as a pedestrian, a vehicle, an animal, or the like. The spatial state change relationship of the obstacle around the vehicle can be understood as a change relationship of the spatial position of the obstacle around the vehicle with respect to the door 40.

In some embodiments, the controller 30 is also configured to receive feedback signals from an actuator of the vehicle door 40, such as a lock status, door speed, door position, current, voltage, anti-pinch signal, etc. It can be appreciated that the feedback control is performed on the vehicle door 40 according to the feedback signal of the executing mechanism of the vehicle door 40, so that the hovering of the intelligent door can be accurately realized, the anti-pinch function can be realized in the opening and closing process, and the functions of emergency avoidance and the like can be rapidly realized.

In some embodiments, the controller 30 is further configured to receive a door control signal, which may include one or more of vehicle speed information, gear information of the vehicle, and received voice information, and the controller 30 is configured to control the door 40 based on the received door control signal.

In one example, the controller 30 is configured to detect vehicle speed information and gear information of the vehicle; when the vehicle speed information and the gear information of the vehicle satisfy the preset conditions, the image information of the obstacle around the vehicle and the ultrasonic information of the obstacle around the vehicle are fused, a spatial state change relation between the obstacle around the vehicle and the door 40 is obtained, and the door 40 is controlled according to the spatial state change relation. Thus, the control of the vehicle door 40 is ensured in time, the control is more intelligent, and the user experience is improved. The preset conditions may include a vehicle speed of less than or equal to a preset speed (e.g., 3 km/h) and a gear being a preset gear (e.g., neutral, park). It will be appreciated that when the vehicle speed is greater than the preset speed or the gear is the forward gear or the reverse gear, the user may not have the intention to open or close the door, and at this time, the spatial state change relationship is not required to be determined, and the vehicle door 40 is not required to be controlled according to the spatial state change relationship.

In another example, the controller 30 is configured to analyze the received user voice to determine whether the user voice includes a door opening/closing command, and in case of receiving the door opening/closing command, perform fusion processing on image information of an obstacle around the vehicle and ultrasonic information of the obstacle around the vehicle, obtain a spatial state change relationship between the obstacle around the vehicle and the door 40, and control the door 40 according to the spatial state change relationship.

It will be appreciated that the controller 30 may be used for intelligent obstacle recognition, door pinch prevention detection, door speed detection, door position detection, door opening and closing intention detection, emergency risk avoidance triggering, alarm protection control, central control liquid crystal display intelligent door system display control, and the like.

In one example, as shown in fig. 2, a flow of the fusion processing of the image information of the obstacle around the vehicle and the ultrasonic information of the obstacle around the vehicle is performed, and the spatial state change relationship between the obstacle around the vehicle and the door 40 can be determined by processing the image information of the obstacle around the vehicle and the ultrasonic information of the obstacle around the vehicle acquired at a plurality of times.

In some embodiments of the present invention, the controller 30 is further configured to establish a spatial rectangular coordinate system according to a positional relationship between the vehicle-mounted camera 10 and the vehicle-mounted radar 20, determine three-dimensional coordinates of the vehicle-surrounding obstacle in the spatial rectangular coordinate system according to image information of the vehicle-surrounding obstacle and ultrasonic information of the vehicle-surrounding obstacle, and determine a spatial state change relationship between the vehicle-surrounding obstacle and the door 40 according to the three-dimensional coordinates of the vehicle-surrounding obstacle in the spatial rectangular coordinate system.

In this way, the spatial position of the obstacle around the vehicle with respect to the door 40 can be determined more accurately.

Specifically, the spatial state change relationship may be divided based on different spatial regions, so that after determining the three-dimensional coordinates of the obstacle around the vehicle, the spatial region in which the obstacle around the vehicle is located can be determined, and thus the spatial state change relationship of the obstacle around the vehicle can be rapidly determined.

In some embodiments of the present invention, the vehicle-mounted radar 20 includes a first radar and a second radar, and the vehicle-mounted camera 10, the first radar and the second radar are located in the same straight line, and the controller 30 is further configured to establish a space rectangular coordinate system with the vehicle-mounted camera 10 as an origin and the straight line as an x-axis.

In this way, it is convenient to determine the three-dimensional coordinates of the obstacle around the vehicle.

Specifically, please refer to fig. 3, in which the vehicle-mounted camera O is taken as an origin, a straight line where the vehicle-mounted camera O, the first radar L1 and the second radar L2 are located is taken as an x-axis, the x-axis is substantially parallel to a horizontal plane, a straight line which passes through the vehicle-mounted camera O and is perpendicular to the horizontal plane and faces upwards is taken as a y-axis, and a straight line which passes through the vehicle-mounted camera O and is parallel to the horizontal plane and is perpendicular to the x-axis is taken as a z-axis, so that a space rectangular coordinate system is established.

In some embodiments of the present invention, the three-dimensional coordinates of the obstacle around the vehicle include a first x-axis component, a first y-axis component, and a z-axis component, wherein the controller 30 is further configured to determine a projection length of a line of the first radar with the obstacle around the vehicle on the x-axis based on the ultrasonic information acquired by the first radar and the ultrasonic information acquired by the second radar, and determine the first x-axis component based on a distance between the first radar and the in-vehicle camera 10 and the projection length, and determine the two-dimensional coordinates of the obstacle around the vehicle in the two-dimensional imaging image based on the image information, and determine the first y-axis component and the z-axis component based on the first x-axis component and the two-dimensional coordinates, and use the first x-axis component, the first y-axis component, and the z-axis component as the three-dimensional coordinates of the obstacle around the vehicle.

Referring again to fig. 3, since the positions of the in-vehicle camera O, the first radar L1, and the second radar L2 have been fixed in advance, the distance a1 between the in-vehicle camera O and the first radar L1 is a known amount, and the distance a2 between the first radar L1 and the second radar L2 is also a known amount. The distance b1 of the first radar L1 from the obstacle W around the vehicle can be determined from the ultrasonic information acquired by the first radar L1. The distance b2 of the second radar L2 from the obstacle W around the vehicle can be determined from the ultrasonic information acquired by the second radar L2.

In the triangle surrounded by the first radar L1, the second radar L2, and the obstacle W around the vehicle, since the three-side lengths a2, b1, and b2 of the triangle are all known amounts, the projection length a3 of the line L1W of the first radar L1 and the obstacle W around the vehicle on the x-axis, that is, the length of the line segment L1H can be calculated based on the pythagorean theorem.

The sum of the distance a1 between the first radar L1 and the in-vehicle camera O and the projection length a3 is taken as a first x-axis component x1.

A two-dimensional imaging image can be understood as a two-dimensional image generated from an external light signal, i.e., image information, received by the in-vehicle camera 10 in the actual shooting size range. When the vehicle surrounding obstacle is within the shooting size range of the in-vehicle camera 10 in the real space, the vehicle surrounding obstacle may be present in the two-dimensional imaging image, and the corresponding two-dimensional coordinates of the vehicle surrounding obstacle in the two-dimensional imaging image are known. The dimensions of the two-dimensional imaging image include a first long side and a first wide side, wherein the first long side corresponds to a horizontal plane and the first wide side is perpendicular to the first long side. And establishing an x-axis according to the first long side of the two-dimensional imaging image, and establishing a y-axis according to the first wide side of the two-dimensional imaging image, namely determining a second x-axis component x0 and a second y-axis component y0 of the obstacle around the vehicle according to the position of the obstacle around the vehicle in the two-dimensional imaging image, so as to determine the two-dimensional coordinates of the obstacle around the vehicle as (x 0, y 0). In some embodiments, the size of the two-dimensional imaging image is the size of the display screen.

In some embodiments of the present invention, the two-dimensional coordinates include a second x-axis component and a second y-axis component, wherein the controller 30 is further configured to determine a proportional relationship between the shooting size of the in-vehicle camera 10 and the size of the two-dimensional imaging image based on the first x-axis component and the second x-axis component, determine the first y-axis component based on the proportional relationship and the second y-axis component, and determine the z-axis component based on the proportional relationship.

It will be appreciated that the shooting size of the in-vehicle camera 10 includes a second long side and a second wide side, wherein the second long side corresponds to a horizontal plane, and the second wide side is perpendicular to the second long side. In the case where the parameters and the installation position of the in-vehicle camera 10 are determined, the shooting size of the in-vehicle camera 10 has a proportional relationship with the distance of the obstacle around the vehicle to the x-axis and the size of the two-dimensional imaging image. For example, in the case where the parameters and the installation position of the in-vehicle camera 10 are determined, when the size of the display screen, that is, the size of the two-dimensional imaging image is 8 inches, an external environment 5 meters away from the x-axis is photographed, and a range of the photographing size of 13×9.8 meters is displayed as a full screen on the display screen; the shooting distance is 10 meters from the external environment, and the range of the shooting size of 26 x 19.5 meters is displayed on the display screen to be full. Accordingly, the proportional relationship between the shooting size of the in-vehicle camera 10 and the size of the two-dimensional imaging image can be determined from the ratio of the first x-axis component and the second x-axis component.

Further, after determining the proportional relationship between the shooting size of the in-vehicle camera 10 and the size of the two-dimensional imaging image, the first y-axis component y1 and the z-axis component z1 may be determined according to the proportional relationship, thereby determining the three-dimensional coordinates of the obstacle around the vehicle as (x 1, y1, z 1).

In some embodiments of the present invention, the controller 30 is further configured to determine three-dimensional coordinates of the door 40 in the space rectangular coordinate system, calculate a distance between the three-dimensional coordinates of the obstacle around the vehicle in the space rectangular coordinate system and the three-dimensional coordinates of the door 40 in the space rectangular coordinate system, and determine a spatial state change relationship between the obstacle around the vehicle and the door 40 according to the distance.

In this way, the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 are determined based on the same space rectangular coordinate system, so that the distance between the obstacle around the vehicle and the door 40 is determined according to the three-dimensional coordinates, and the space state change relationship between the obstacle around the vehicle and the door 40 can be determined.

Specifically, it can be confirmed according to the installation position of the door 40The three-dimensional coordinates of the door 40 are determined. In some embodiments, the coordinates of the center point of the door frame may be taken as the three-dimensional coordinates of the door 40. In one example, the three-dimensional coordinates of the obstacle around the vehicle are (x 1, y1, z 1), and the three-dimensional coordinates of the door 40 are (x 2, y2, z 2), and the distance D between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 can be calculated by solving the following formula: d (D) ² ＝(x1-x2) ² +(y1-y2) ² +(z1-z2) ² 。

In some embodiments of the present invention, the controller 30 is further configured to determine that the obstacle around the vehicle is in a hazardous space of the door 40 when the distance is less than the first threshold; determining that an obstacle around the vehicle is in a safe space of the door 40 when the distance is greater than the second threshold; when the distance is equal to or greater than the first threshold value and equal to or less than the second threshold value, it is determined that the obstacle around the vehicle is in the critical space of the door 40, and the first threshold value is smaller than the second threshold value.

In this way, after the three-dimensional coordinates of the vehicle surrounding obstacle and the three-dimensional coordinates of the door 40 are determined, by calculating the distance between the vehicle surrounding obstacle and the door 40, the spatial state change relationship of the vehicle surrounding obstacle with respect to the door 40 can be quickly determined.

In particular, the spatial state change relationship may include a hazardous space at the door 40, a safe space at the door 40, and a critical space at the door 40.

The dangerous space of the door 40 is understood to be the whole travel space area of the door 40 during the opening and closing of the door, and the vehicle surrounding obstacle in this space area is an effective vehicle surrounding obstacle, and the opening and closing of the door cannot be performed regardless of whether the vehicle surrounding obstacle is in a moving state or a stationary state.

The safe space of the door 40 is understood to be a space area where the opening and closing of the door is not affected, and in this space area, the surrounding obstacle of the vehicle is an ineffective surrounding obstacle of the vehicle, and it is not necessary to consider whether the surrounding obstacle of the vehicle is in a moving state or a stationary state.

The critical space of the door 40, that is, the space region between the safe space and the dangerous space, in which the surrounding obstacle of the vehicle has uncertainty, can be determined as a valid surrounding obstacle if the surrounding obstacle of the space region has a movement tendency toward the dangerous space, and can be determined as an invalid surrounding obstacle if the surrounding obstacle of the space region has a movement tendency toward the safe space.

In one example, the first threshold value is 1m and the second threshold value is 2m, that is, when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is less than 1m, it may be determined that the obstacle around the vehicle is in the dangerous space of the door 40; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is greater than 2m, it may be determined that the obstacle around the vehicle is in the safety space of the door 40; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is 1m or more and 2m or less, it may be determined that the obstacle around the vehicle is in the critical space of the door 40.

In certain embodiments, the first and second thresholds may include a tolerance range (e.g., ±20cm, ±30 cm). Thus, the safety of the vehicle door when being opened and closed can be further improved. For example, when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is less than 1m±20cm, it may be determined that the obstacle around the vehicle is in a dangerous space of the door 40; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is greater than 2m±20cm, it may be determined that the obstacle around the vehicle is in the safe space of the door 40; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door 40 is 1m±20cm or more and 2m±20cm or less, it may be determined that the obstacle around the vehicle is in the critical space of the door 40.

In some embodiments of the present invention, the controller 30 is further configured to control the door 40 to prohibit execution of the received door opening and closing command when the obstacle around the vehicle is in the dangerous space of the door 40 and the door 40 is in the stopped state; when the obstacle around the vehicle is in the dangerous space of the door 40 and the door 40 is in a moving state, the door 40 is controlled to stop moving and anti-collision protection is carried out; when the obstacle around the vehicle is in the safety space of the door 40 and the door 40 is in a stopped state, controlling the door 40 to execute the received door opening and closing command; when the obstacle around the vehicle is in the safety space of the door 40 and the door 40 is in a moving state, controlling the door 40 to execute the received door opening and closing command; when the obstacle around the vehicle is in the critical space of the door 40 and the door 40 is in a stopped state, controlling the door 40 to delay executing the received door opening and closing instruction and performing anti-collision protection; when the vehicle surrounding obstacle is in the critical space of the door 40 and the door 40 is in a moving state, the door 40 is controlled according to a movement time period required for the vehicle surrounding obstacle to move to the dangerous space of the door 40.

In this way, the spatial state change relationship and the state of the door 40 are comprehensively considered, and the door control is performed according to the spatial state change relationship and the state of the door 40, so that the opening and closing of the door 40 can be more intelligent, and the safety is improved. It can be understood that in the related art, the door opening or closing operation is performed immediately after the door opening or closing instruction is received, if the door opening or closing instruction is issued by the user under the condition that the user does not notice and observe the front and rear vehicles and pedestrians, the door opening or closing instruction may collide with the pedestrians or the vehicles at this time, so as to cause serious traffic accidents.

Specifically, the stopped state may be a state in which the door 40 remains stationary after being fully opened, a state in which the door 40 remains stationary after being fully closed, or a state in which the door 40 is stationary at any one of the travel positions between the fully opened and the fully closed. The movement state may be a state in which the door 40 is gradually opened or a state in which the door 40 is gradually closed. The movement state of the door 40 may be generated by automatic movement of the door 40 according to a door opening and closing command, or may be generated by manual operation of the door 40 by a user.

Further, when the obstacle around the vehicle is in the dangerous space of the door 40 and the door 40 is in the stopped state, if the door opening/closing instruction is detected, it is not executed.

When the obstacle around the vehicle is in the dangerous space of the door 40, if the door opening or closing action is being performed, the door opening or closing action is immediately stopped and anti-collision protection is performed, so that the injury to the personal safety and property of the user caused by the continued opening or closing of the door 40 is prevented.

When the obstacle around the vehicle is in the safe space of the door 40, the door 40 can be normally opened and closed according to the received door opening and closing command, regardless of whether the door 40 is in a stopped state or a moving state.

When the door 40 is in the stopped state and the obstacle around the vehicle is in the critical space of the door 40, if a door opening/closing command is received, that is, if the user intends to open/close the door, the door opening/closing command is not executed, and the collision avoidance is performed at the same time, and after waiting for the obstacle around the vehicle to enter the safety space, the door opening/closing command is executed.

When the vehicle surrounding obstacle is in the critical space of the door 40, if a door opening or closing action is being performed, it is possible to further determine how to control the door 40 in combination with the movement time period required for the vehicle surrounding obstacle to move to the dangerous space of the door 40.

In some embodiments of the present invention, the controller 30 is further configured to control the door 40 to stop moving and perform crash protection when the door 40 is executing a door closing command and the movement duration is less than or equal to the remaining closing duration of the door 40; when the door 40 is executing the door closing command and the movement time period is longer than the remaining closing time period of the door 40, the door 40 is controlled to continue executing the door closing command.

Therefore, in the process of closing the vehicle door, the door closing instruction is ensured to be normally completed as much as possible on the premise of ensuring the safety, so that waiting of a user caused by incapability of closing the door is avoided, and delay of schedule of the user is prevented.

Specifically, referring to fig. 4, it is possible to estimate whether the vehicle surrounding obstacle in the image information is the same vehicle surrounding obstacle or not, and the movement tendency of the vehicle surrounding obstacle with respect to the door 40, based on the ultrasonic information of the vehicle surrounding obstacle acquired by the vehicle-mounted radar 20 at different times. For example, if the distance from the vehicle to the obstacle around the vehicle detected by the radar at different times is unchanged, the obstacle around the vehicle may be considered to be in a stationary state; if the distance from the vehicle to the obstacle around the vehicle detected by the radar gradually decreases, the movement tendency of the obstacle around the vehicle can be considered as approaching the door 40; if the distance from the vehicle to the obstacle around the vehicle detected by the radar gradually increases, the movement tendency of the obstacle around the vehicle can be considered to be away from the door 40.

Referring to fig. 5, from the image information of the surrounding obstacle of the vehicle acquired by the in-vehicle camera 10 at different times, it is possible to estimate whether the surrounding obstacle of the vehicle in the image information is the same surrounding obstacle of the vehicle and the movement trend of the surrounding obstacle of the vehicle with respect to the door 40. For example, a plurality of sets of two-dimensional imaging images can be acquired through a plurality of vehicle-mounted cameras 10 with different mounting positions, image analysis is performed on the plurality of sets of two-dimensional imaging images, each set of two-dimensional imaging images is divided into a plurality of grids, points are drawn on the outline of the obstacle around the vehicle in each set of two-dimensional imaging images, and if the area of the outline of the obstacle around the vehicle is unchanged, the obstacle around the vehicle can be considered to be in a static state; if the area of the surrounding obstacle outline of the vehicle becomes gradually larger, the movement tendency of the surrounding obstacle of the vehicle can be considered as approaching the door 40; if the area of the surrounding obstacle outline of the vehicle becomes smaller gradually, the movement tendency of the surrounding obstacle of the vehicle can be considered to be away from the door 40.

Further, the movement time period required for the movement of the obstacle around the vehicle to the dangerous space of the door 40 can be calculated in combination with the movement trend of the obstacle around the vehicle and the estimated movement speed of the obstacle around the vehicle. The current position of the door 40 and the closing speed of the door 40 may be determined by the related sensor data and the current data, and thus the remaining closing time period of the door 40 may be determined according to the current position of the door 40 and the closing speed of the door 40. If the door 40 is executing a door closing command and the movement time required for the vehicle surrounding obstacle to move to the dangerous space of the door 40 is less than or equal to the remaining closing time of the door 40, continued closing of the door 40 may cause the vehicle surrounding obstacle to collide with the door 40 or the vehicle surrounding obstacle to be caught by the door 40, and thus, the door 40 should be controlled to stop moving and be crash-protected. If the door 40 is executing a door closing command and the movement time required for the vehicle surrounding obstacle to move to the dangerous space of the door 40 is longer than the remaining closing time of the door 40, continued closing of the door 40 does not cause the vehicle surrounding obstacle to collide with the door 40 or the vehicle surrounding obstacle to be caught by the door 40, and thus, the door 40 can be controlled to continue executing the door closing command.

In some embodiments of the present invention, the crash protection may include one or more of alarm prompts, locking the seat belt, locking the door 40, and opening and closing the door in the reverse direction.

Specifically, referring to FIG. 1, a vehicle door control system 100 may include an alarm protection device 50, with the alarm protection device 50 being coupled to a controller 30. The alarm protection device 50 may include a turn signal, a horn, a safety belt, etc. The alarm prompts may include one or more of audible alarms, light alarms, text prompts. The audible alarm may be implemented by a horn (whistling) or a speaker (voice broadcasting prompt), etc. The light warning may alert the user in the vehicle and/or other pedestrians or vehicles in the surrounding environment of the vehicle by way of a turn signal or other lights. The text prompt can be realized through the display screen inside and outside the vehicle.

In the door opening and closing process, if potential safety hazards exist in the surrounding environment of the vehicle, the safety belt and/or the vehicle door 40 can be locked forcibly, a user is prevented from opening and closing the door, and the safety belt and the vehicle door 40 are unlocked after the surrounding environment confirms safety, so that accidents are prevented, and the safety of the user is ensured.

In some embodiments of the present invention, the door control system 100 further includes a display 60, where the display 60 is connected to the controller 30, and the controller 30 is further configured to control the display 60 to dynamically display the spatial state change relationship.

Therefore, the spatial state change relation of the barriers around the vehicle becomes visible, is more vivid and is beneficial to improving user experience.

Specifically, the spatial state change relationship may be dynamically displayed by way of the display 60 (as shown in fig. 6) in a manner that displays the door panoramic visualization system and the simulated three-dimensional stereoscopic environment visualization system. In this way, during operation of the door 40, a user can timely learn about obstacles around the vehicle through the display 60, and can protect himself at the first time once danger is found, and manually take anti-collision protection measures.

In some embodiments, the vehicle may wirelessly connect and communicate with the server, and then the vehicle may download road data, building data, etc. surrounding the current vehicle from the server and fuse the acquired road data and building data into the door panorama visual system, thereby further increasing the authenticity of the door panorama visual system.

In some embodiments, the controller 30 is further configured to perform door control based on the ultrasonic information of the obstacle around the vehicle collected by the vehicle radar 20 in the case where the ambient light is dim, the vehicle-mounted camera 10 is covered by a stain, or the vehicle is rainy or snowy. Also, at this time, the speed at which the door 40 is opened or closed may be reduced, thereby preventing the door 40 from colliding with an obstacle around the vehicle in the blind area.

In some embodiments, the controller 30 is further configured to perform door control based on the image information of the surrounding obstacle of the vehicle acquired by the vehicle-mounted camera 10 when the surrounding obstacle of the vehicle is in the blind detection zone of the vehicle-mounted radar 20. When it is detected that the obstacle around the vehicle is in the dangerous space, the opening and closing operation of the door 40 is not performed, and an alarm prompt is made. And if the door is opened or closed, immediately executing emergency danger avoidance and performing anti-collision protection.

Referring to fig. 7, the present invention provides a vehicle door control method, which includes:

s11: acquiring image information of obstacles around a vehicle, and acquiring ultrasonic information of the obstacles around the vehicle;

s13: the method comprises the steps of performing fusion processing on image information of surrounding obstacles of a vehicle and ultrasonic information of the surrounding obstacles of the vehicle to obtain a space state change relation between the surrounding obstacles of the vehicle and a vehicle door;

s15: and controlling the vehicle door according to the space state change relation.

According to the vehicle door control method, the vehicle door is controlled according to the spatial state change relation of the surrounding obstacles of the vehicle relative to the vehicle door, so that the intelligent degree of the vehicle door is improved, and the safety and reliability of the vehicle door are improved.

Moreover, the door control scheme in the related art is focused only on the realization of automatic door opening and automatic door closing, and does not recognize obstacles around the vehicle and perform automatic door opening and automatic door closing according to a spatial state change relationship of the obstacles around the vehicle. That is, the door control scheme in the related art has problems of low degree of intelligence, low reliability, and the like.

In the vehicle door control method of the embodiment, the vehicle-mounted radar has the advantage of being less influenced by the environment in the working process, for example, the vehicle-mounted radar can detect the obstacle around the vehicle at night or in a dark place, the vehicle-mounted camera has the advantage of being capable of realizing non-blind area detection by means of the mounting position, the image information collected by the vehicle-mounted camera and the ultrasonic information collected by the vehicle-mounted radar are subjected to fusion analysis, and the advantages of the two sides are combined to compensate the disadvantages of the two sides, so that the accuracy of identifying the obstacle around the vehicle is ensured, the vehicle door control method can be realized, and the related scheme for judging the spatial state change relation between the obstacle around the vehicle and the vehicle door is added in the vehicle door control method, so that the vehicle door control method of the embodiment is more intelligent, reliable and safe. Meanwhile, the vehicle door control method based on the vehicle-mounted camera and the vehicle-mounted radar can effectively control cost.

In particular, the door may include a side-hung door, a side-by-side door, a side-slip door, and the like.

The vehicle-mounted cameras can comprise a plurality of vehicle-mounted cameras, and the directions of the vehicle-mounted cameras can be different, so that the environment around the vehicle is monitored in multiple directions, and the image information of obstacles around the vehicle is acquired. The plurality of vehicle-mounted cameras can be distributed around the vehicle body of the vehicle or can be gathered at the top of the vehicle. When a plurality of in-vehicle cameras gather at the top of vehicle, a plurality of cameras can be set to be fixedly installed at the top of vehicle, also can be set to be rotatably installed at the top of vehicle, and a plurality of in-vehicle cameras can be set to be rotated synchronously, also can be set to be rotated alone.

The vehicle-mounted camera may also include one, and one vehicle-mounted camera may be configured to be rotatably mounted on top of the vehicle, thereby achieving the effect of monitoring the environment surrounding the vehicle. The vehicle-mounted camera can shoot the surrounding obstacles of the vehicle in the self-facing range according to the preset sampling frequency to acquire the image information of the surrounding obstacles of the vehicle, and can shoot the surrounding obstacles of the vehicle in the self-facing range in real time in a video mode to acquire the image information of the surrounding obstacles of the vehicle.

The vehicle-mounted radar can comprise a plurality of vehicle-mounted radars, and the directions of the vehicle-mounted radars can be different, so that the environment around the vehicle is detected in multiple directions, and the ultrasonic information of obstacles around the vehicle is acquired.

The obstacle around the vehicle may be a person or object movable around the vehicle, such as a pedestrian, a vehicle, an animal, or the like. The spatial state change relationship of the obstacle around the vehicle can be understood as a change relationship of the spatial position of the obstacle around the vehicle with respect to the door.

In certain embodiments, prior to step S11, the door control method further comprises: detecting whether a door opening and closing instruction is received; when the door opening/closing command is received, the process advances to step S11, step S13, and step S15. Thus, the operation space is saved, and the power consumption is reduced. The door open/close command may be used to indicate that the door is open or to indicate that the door is closed. In some embodiments, the vehicle may include a voice recognition component that is capable of analyzing received user speech to determine whether door opening and closing instructions are included in the user speech. In some embodiments, the vehicle may include an input assembly on which a user can select to open and close the door, thereby generating door open and close commands.

In certain embodiments, prior to step S11, the door control method further comprises: detecting speed information and gear information of a vehicle; when the vehicle speed information and the shift position information of the vehicle satisfy the preset conditions, the process proceeds to step S11, step S13, and step S15. Therefore, the vehicle door control method is guaranteed to be executed in time, so that the vehicle door control method is more intelligent, and the user experience is improved. Specifically, the preset conditions may include that the vehicle speed is equal to or less than a preset speed (e.g., 3 km/h), and that the gear is a preset gear (e.g., neutral, park). It is understood that when the vehicle speed is greater than the preset speed or the gear is the forward gear or the reverse gear, the user may not have the intention to open or close the door, and it is not necessary to proceed to step S11, step S13 and step S15.

In one example, as shown in fig. 2, a flow of the fusion processing of the image information of the obstacle around the vehicle and the ultrasonic information of the obstacle around the vehicle is performed, and the spatial state change relationship between the obstacle around the vehicle and the door can be determined by processing the image information of the obstacle around the vehicle and the ultrasonic information of the obstacle around the vehicle acquired at a plurality of times.

Referring to fig. 8, in some embodiments of the present invention, step S13 includes:

S131: establishing a space rectangular coordinate system according to the position relation between the vehicle-mounted camera and the vehicle-mounted radar;

s133: determining three-dimensional coordinates of the surrounding obstacles in the space rectangular coordinate system according to the image information of the surrounding obstacles and the ultrasonic information of the surrounding obstacles;

s135: and determining the spatial state change relation between the surrounding obstacles and the vehicle door according to the three-dimensional coordinates of the surrounding obstacles in the space rectangular coordinate system.

In this way, the spatial position of the obstacle around the vehicle with respect to the door can be determined more accurately.

In some embodiments of the present invention, the vehicle-mounted radar includes a first radar and a second radar, and the vehicle-mounted camera, the first radar and the second radar are located on the same straight line, and step S131 includes: and establishing a space rectangular coordinate system by taking the vehicle-mounted camera as an origin and taking a straight line as an x axis.

Referring to fig. 9, in some embodiments of the present invention, the three-dimensional coordinates of the obstacle around the vehicle include a first x-axis component, a first y-axis component, and a z-axis component, and step S133 includes:

s1331: determining the projection length of a connecting line of the first radar and obstacles around the vehicle on an x-axis according to the ultrasonic information acquired by the first radar and the ultrasonic information acquired by the second radar;

s1332: determining a first x-axis component according to the distance between the first radar and the vehicle-mounted camera and the projection length;

s1333: determining two-dimensional coordinates of obstacles around the vehicle in the two-dimensional imaging image according to the image information;

s1334: determining a first y-axis component and a z-axis component from the first x-axis component and the two-dimensional coordinates;

s1335: the first x-axis component, the first y-axis component, and the z-axis component are taken as three-dimensional coordinates of an obstacle around the vehicle.

In step S1331, in the triangle surrounded by the first radar L1, the second radar L2, and the obstacle W around the vehicle, since the three-side lengths a2, b1, and b2 of the triangle are all known amounts, the projection length a3 of the line L1W of the first radar L1 and the obstacle W around the vehicle on the x-axis, that is, the length of the line segment L1H, can be calculated based on the pythagorean theorem.

In step S1332, the sum of the distance a1 between the first radar L1 and the in-vehicle camera O and the projection length a3 is taken as the first x-axis component x1.

In step S1333, the two-dimensional imaging image can be understood as a two-dimensional image generated from the image information, which is an external light signal received by the in-vehicle camera within the actual shooting size range. When the vehicle surrounding obstacle is within the shooting size range of the in-vehicle camera in the real space, the vehicle surrounding obstacle may be present in the two-dimensional imaging image, and the corresponding two-dimensional coordinates of the vehicle surrounding obstacle in the two-dimensional imaging image are known. The dimensions of the two-dimensional imaging image include a first long side and a first wide side, wherein the first long side corresponds to a horizontal plane and the first wide side is perpendicular to the first long side. And establishing an x-axis according to the first long side of the two-dimensional imaging image, and establishing a y-axis according to the first wide side of the two-dimensional imaging image, namely determining a second x-axis component x0 and a second y-axis component y0 of the obstacle around the vehicle according to the position of the obstacle around the vehicle in the two-dimensional imaging image, so as to determine the two-dimensional coordinates of the obstacle around the vehicle as (x 0, y 0). In some embodiments, the size of the two-dimensional imaging image is the size of the display screen.

Referring to fig. 10, in some embodiments of the present invention, the two-dimensional coordinates include a second x-axis component and a second y-axis component, and step S1334 includes:

s13341: determining a proportional relation between the shooting size of the vehicle-mounted camera and the size of the two-dimensional imaging image according to the first x-axis component and the second x-axis component;

s13342: determining a first y-axis component according to the proportional relationship and the second y-axis component;

s13343: the z-axis component is determined from the proportional relationship.

It can be appreciated that the shooting size of the vehicle-mounted camera includes a second long side and a second wide side, wherein the second long side corresponds to the horizontal plane, and the second wide side is perpendicular to the second long side. Under the condition that parameters and installation positions of the vehicle-mounted cameras are determined, the shooting size of the vehicle-mounted cameras is in proportional relation with the distance from the obstacle around the vehicle to the x-axis and the size of the two-dimensional imaging image. For example, in the case where parameters and installation positions of the in-vehicle camera are determined, when the size of the display screen, that is, the size of the two-dimensional imaging image is 8 inches, an external environment 5 m away from the x-axis is photographed, and a range of the photographing size of 13 x 9.8 m is displayed as a full screen on the display screen; the shooting distance is 10 meters from the external environment, and the range of the shooting size of 26 x 19.5 meters is displayed on the display screen to be full. Accordingly, a proportional relationship between the photographing size of the in-vehicle camera and the size of the two-dimensional imaging image can be determined from the ratio of the first x-axis component and the second x-axis component.

Further, after determining the proportional relationship between the shooting size of the in-vehicle camera and the size of the two-dimensional imaging image, the first y-axis component y1 and the z-axis component z1 may be determined according to the proportional relationship, thereby determining that the three-dimensional coordinates of the obstacle around the vehicle are (x 1, y1, z 1).

Referring to fig. 11, in some embodiments of the present invention, step S135 includes:

s1351: determining three-dimensional coordinates of the vehicle door in a space rectangular coordinate system, and calculating the distance between the three-dimensional coordinates of the obstacle around the vehicle in the space rectangular coordinate system and the three-dimensional coordinates of the vehicle door in the space rectangular coordinate system;

s1353: a spatial state change relationship between an obstacle around the vehicle and the door is determined based on the distance.

Therefore, the three-dimensional coordinates of the surrounding obstacles and the three-dimensional coordinates of the vehicle door are determined based on the same space rectangular coordinate system, so that the distance between the surrounding obstacles and the vehicle door can be conveniently determined according to the three-dimensional coordinates, and further the space state change relation between the surrounding obstacles and the vehicle door can be determined.

Specifically, the three-dimensional coordinates of the door may be determined according to the installation position of the door. In some embodiments, the coordinates of the center point of the door frame may be taken as the three-dimensional coordinates of the door. In one example, the three-dimensional coordinates of the obstacle around the vehicle are (x 1, y1, z 1), The three-dimensional coordinates of the door are (x 2, y2, z 2), and the distance D between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door can be calculated by solving the following formula: d (D) ² ＝(x1-x2) ² +(y1-y2) ² +(z1-z2) ² 。

Referring to fig. 12, in some embodiments of the present invention, step S1353 includes:

s13531: determining that an obstacle around the vehicle is in a dangerous space of the vehicle door when the distance is smaller than a first threshold value;

s13532: determining that an obstacle around the vehicle is in a safe space of the vehicle door when the distance is greater than a second threshold value;

s13533: and when the distance is greater than or equal to a first threshold value and less than or equal to a second threshold value, determining that the obstacle around the vehicle is in the critical space of the vehicle door, wherein the first threshold value is smaller than the second threshold value.

In this way, after the three-dimensional coordinates of the vehicle surrounding obstacle and the three-dimensional coordinates of the door are determined, by calculating the distance between the vehicle surrounding obstacle and the door, the spatial state change relationship of the vehicle surrounding obstacle with respect to the door can be quickly determined.

In particular, the spatial state change relationship may include a hazardous space at the door, a safe space at the door, and a critical space at the door.

The dangerous space of the door is understood to be the whole travel space area of the door during the opening and closing of the door, and the surrounding obstacle of the vehicle in the space area is an effective surrounding obstacle of the vehicle, and the opening and closing of the door can not be performed no matter the surrounding obstacle of the vehicle is in a moving state or a static state.

The safe space of the door is understood to be a space area that does not affect the opening and closing of the door, in which the surrounding obstacle of the vehicle is an ineffective surrounding obstacle of the vehicle, and it is not necessary to care whether the surrounding obstacle of the vehicle is in a moving state or a stationary state.

The critical space of the door, that is, the space region between the safe space and the dangerous space, the surrounding obstacle of the vehicle in the space region has uncertainty, and if the surrounding obstacle of the space region has a movement tendency toward the dangerous space, it can be determined as a valid surrounding obstacle of the vehicle, and if the surrounding obstacle of the space region has a movement tendency toward the safe space, it can be determined as an invalid surrounding obstacle of the vehicle.

In one example, the first threshold is 1m, and the second threshold is 2m, that is, when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is less than 1m, it may be determined that the obstacle around the vehicle is in the dangerous space of the door; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is greater than 2m, it can be determined that the obstacle around the vehicle is in the safety space of the door; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is 1m or more and 2m or less, it may be determined that the obstacle around the vehicle is in the critical space of the door.

In certain embodiments, the first and second thresholds may include a tolerance range (e.g., ±20cm, ±30 cm). Thus, the safety of the vehicle door when being opened and closed can be further improved. For example, when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is less than 1m±20cm, it may be determined that the obstacle around the vehicle is in a dangerous space of the door; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the vehicle door is more than 2 m+/-20 cm, determining that the obstacle around the vehicle is in a safety space of the vehicle door; when the distance between the three-dimensional coordinates of the obstacle around the vehicle and the three-dimensional coordinates of the door is 1m±20cm or more and 2m±20cm or less, it can be determined that the obstacle around the vehicle is in the critical space of the door.

Referring to fig. 13, in some embodiments of the present invention, step S15 includes:

s151: when the obstacle around the vehicle is in the dangerous space of the vehicle door and the vehicle door is in a stop state, controlling the vehicle door to prohibit execution of the received door opening and closing command;

s152: when the obstacle around the vehicle is in the dangerous space of the vehicle door and the vehicle door is in a moving state, controlling the vehicle door to stop moving and performing anti-collision protection;

S153: when the obstacle around the vehicle is in the safety space of the vehicle door and the vehicle door is in a stop state, controlling the vehicle door to execute the received door opening and closing instruction;

s154: when the obstacle around the vehicle is in the safety space of the vehicle door and the vehicle door is in a moving state, controlling the vehicle door to execute the received door opening and closing instruction;

s155: when the obstacle around the vehicle is in the critical space of the vehicle door and the vehicle door is in a stop state, controlling the vehicle door to delay executing the received door opening and closing instruction and performing anti-collision protection;

s156: when the obstacle around the vehicle is in the critical space of the door and the door is in a moving state, the door is controlled according to the movement time required for the obstacle around the vehicle to move to the dangerous space.

Therefore, the space state change relation and the state of the vehicle door are comprehensively considered, the vehicle door is controlled according to the space state change relation and the state of the vehicle door, the opening and closing of the vehicle door can be more intelligent, and meanwhile safety is improved. It can be understood that in the related art, the door opening or closing operation is performed immediately after the door opening and closing instruction is received, if the door opening and closing instruction is issued by the user under the condition that the user does not notice and observe the front and rear vehicles and pedestrians, the direct door opening may collide with the pedestrians or the vehicles at this time, and serious traffic accidents are caused.

Specifically, the stopped state may be a state in which the door is kept stationary after being fully opened, a state in which the door is kept stationary after being fully closed, or a state in which the door is stationary at any one of the travel positions between the fully opened and the fully closed. The movement state may be a state in which the door is gradually opened or a state in which the door is gradually closed. The motion state of the door may be generated by automatic motion of the door according to a door opening and closing command, or may be generated by manual operation of the door by a user.

Further, step S151 may be understood as not executing if the door opening/closing command is detected when the obstacle around the vehicle is in the dangerous space of the door and the door is in the stopped state. Step S152 may be understood as that if the door opening or closing operation is being performed when the obstacle around the vehicle is in the dangerous space of the door, the door opening or closing operation is immediately stopped and the crash-proof protection is performed, so as to prevent injury to the personal safety and property of the user due to the continued opening or closing of the door.

Step S153 and step S154, that is, when the obstacle around the vehicle is in the safe space of the door, the door can be normally opened and closed according to the received door opening and closing command regardless of whether the door is in the stopped state or the moving state.

Step S155 may be understood as that, when the vehicle surrounding obstacle is in the critical space of the door and the door is in the stopped state, if the door opening/closing command is received, that is, the user intends to open/close the door, the door is not executed, and at the same time, the collision avoidance is performed, and the door opening/closing command is executed after waiting for the vehicle surrounding obstacle to enter the safety space.

Step S156 may be understood as determining how to control the door in further combination with the movement time period required for the surrounding obstacle to move to the dangerous space of the door if the door opening or closing action is being performed while the surrounding obstacle is in the critical space of the door.

In some embodiments of the invention, controlling a vehicle door according to a movement duration required for an obstacle around the vehicle to move to a dangerous space of the vehicle door includes: when the door is executing a door closing instruction and the movement time is less than or equal to the remaining closing time of the door, controlling the door to stop moving and performing anti-collision protection; and when the door is executing the door closing command and the movement time is longer than the remaining closing time of the door, controlling the door to continue executing the door closing command.

Specifically, referring to fig. 4, it is possible to estimate whether the surrounding obstacle of the vehicle in the image information is the same surrounding obstacle of the vehicle and the movement trend of the surrounding obstacle of the vehicle with respect to the door based on the ultrasonic information of the surrounding obstacle of the vehicle acquired by the vehicle-mounted radar at different times. For example, if the distance from the vehicle to the obstacle around the vehicle detected by the radar at different times is unchanged, the obstacle around the vehicle may be considered to be in a stationary state; if the distance between the surrounding obstacle of the vehicle detected by the radar and the vehicle gradually decreases, the movement trend of the surrounding obstacle of the vehicle can be considered to be approaching the vehicle door; if the distance from the vehicle to the obstacle around the vehicle detected by the radar gradually increases, the movement tendency of the obstacle around the vehicle can be considered to be away from the door.

Referring to fig. 5, according to the image information of the surrounding obstacles of the vehicle collected by the vehicle-mounted camera at different times, it is possible to estimate whether the surrounding obstacles of the vehicle in the image information are the same surrounding obstacle of the vehicle and the movement trend of the surrounding obstacles of the vehicle relative to the door. For example, multiple groups of two-dimensional imaging images can be acquired through multiple vehicle-mounted cameras with different mounting positions, image analysis is carried out on the multiple groups of two-dimensional imaging images, each group of two-dimensional imaging images is divided into multiple grids, points are drawn on the outline of the obstacle around the vehicle in each group of two-dimensional imaging images, and if the area of the outline of the obstacle around the vehicle is unchanged, the obstacle around the vehicle can be considered to be in a static state; if the area of the outline of the obstacle around the vehicle gradually becomes larger, the movement trend of the obstacle around the vehicle can be considered to be approaching the vehicle door; if the area of the surrounding obstacle outline of the vehicle becomes smaller, the movement tendency of the surrounding obstacle of the vehicle can be considered to be away from the door.

Further, the movement time period required for the surrounding obstacle to move to the dangerous space of the vehicle door can be calculated by combining the movement trend of the surrounding obstacle and the calculated movement speed of the surrounding obstacle. The current position of the door and the closing speed of the door can be determined through the related sensor data and the current data, and further the residual closing time length of the door can be determined according to the current position of the door and the closing speed of the door. If the door is executing a door closing command and the movement time required for the surrounding obstacle to move to the dangerous space of the door is less than or equal to the remaining closing time of the door, the continued closing of the door can cause the surrounding obstacle to collide with the door or the surrounding obstacle to be clamped by the door, so that the door should be controlled to stop moving and perform anti-collision protection. If the door is executing the door closing command and the movement time required for the vehicle surrounding obstacle to move to the dangerous space of the door is longer than the remaining closing time of the door, the continued closing of the door does not cause the vehicle surrounding obstacle to collide with the door or the vehicle surrounding obstacle to be pinched by the door, so that the door can be controlled to continue executing the door closing command.

In some embodiments of the invention, crash protection may include one or more of alarm prompts, locking safety belts, locking doors, reversing door opening and closing.

In particular, the alert cues may include one or more of audible alerts, light alerts, text cues. The audible alarm may be implemented by a horn (whistling) or a speaker (voice broadcasting prompt), etc. The light warning may alert the user in the vehicle and/or other pedestrians or vehicles in the surrounding environment of the vehicle by way of a turn signal or other lights. The text prompt can be realized through the display screen inside and outside the vehicle.

In the door opening and closing process, if potential safety hazards exist in the surrounding environment of the vehicle, the safety belt and/or the vehicle door can be locked forcibly, a user is fixed, the door opening and closing of the user is prevented, the safety belt and the vehicle door are unlocked after the surrounding environment confirms safety, accordingly accidents are prevented, and the safety of the user is ensured.

Referring to fig. 14, in some embodiments of the present invention, after step S13, the method further includes:

s17: and dynamically displaying the space state change relation.

Specifically, the spatial state change relationship may be dynamically displayed by displaying the vehicle door panoramic visualization system and the simulated three-dimensional stereoscopic environment visualization system via a display (as shown in fig. 6). Therefore, in the working process of the vehicle door, a user can timely know the obstacles around the vehicle through the display, and can protect the user at the first time once danger is found, and anti-collision protection measures are taken artificially.

In certain embodiments, the door control method further comprises: under the conditions of dim ambient light, stain shielding of a vehicle-mounted camera, rain and snow, and the like, the vehicle door control is performed based on the ultrasonic information of the surrounding obstacles of the vehicle, which is acquired by the vehicle-mounted radar. Also, at this time, the speed at which the door is opened or closed can be reduced, thereby preventing the door from colliding with an obstacle around the vehicle in the blind area.

In certain embodiments, the door control method further comprises: and when the obstacle around the vehicle is in the blind detection zone of the vehicle-mounted radar, the vehicle door control is performed based on the image information of the obstacle around the vehicle, which is acquired by the vehicle-mounted camera. When detecting that the obstacle around the vehicle is in the dangerous space, opening and closing operations of the vehicle door are not executed, and alarm prompt is carried out. And if the door is opened or closed, immediately executing emergency danger avoidance and performing anti-collision protection.

In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Furthermore, the terms "first," "second," and the like, as used in embodiments of the present invention, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or as implying any particular number of features in the present embodiment. Thus, a feature of an embodiment of the invention that is defined by terms such as "first," "second," etc., may explicitly or implicitly indicate that at least one such feature is included in the embodiment. In the description of the present invention, the word "plurality" means at least two or more, for example, two, three, four, etc., unless explicitly defined otherwise in the embodiments.

In the present invention, unless explicitly stated or limited otherwise in the examples, the terms "mounted," "connected," and "fixed" as used in the examples should be interpreted broadly, e.g., the connection may be a fixed connection, may be a removable connection, or may be integral, and it may be understood that the connection may also be a mechanical connection, an electrical connection, etc.; of course, it may be directly connected, or indirectly connected through an intermediate medium, or may be in communication with each other, or in interaction with each other. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to specific embodiments.

While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.

Claims

1. A vehicle door control system, comprising:

the vehicle-mounted camera is used for collecting image information of obstacles around the vehicle;

The vehicle-mounted radar is used for collecting ultrasonic information of obstacles around the vehicle;

the controller is respectively connected with the vehicle-mounted camera and the vehicle-mounted radar, and is used for carrying out fusion processing on the image information of the surrounding obstacles of the vehicle and the ultrasonic information of the surrounding obstacles of the vehicle to obtain a space state change relation between the surrounding obstacles of the vehicle and the vehicle door, and controlling the vehicle door according to the space state change relation.

2. The vehicle door control system according to claim 1, wherein the controller is further configured to establish a spatial rectangular coordinate system based on a positional relationship between the vehicle-mounted camera and the vehicle-mounted radar, determine three-dimensional coordinates of the vehicle-surrounding obstacle within the spatial rectangular coordinate system based on image information of the vehicle-surrounding obstacle and ultrasonic information of the vehicle-surrounding obstacle, and determine a spatial state change relationship between the vehicle-surrounding obstacle and the vehicle door based on the three-dimensional coordinates of the vehicle-surrounding obstacle within the spatial rectangular coordinate system.

3. The vehicle door control system of claim 2, wherein the controller is further configured to determine three-dimensional coordinates of the vehicle door within the space rectangular coordinate system, calculate a distance between the three-dimensional coordinates of the vehicle surrounding obstacle within the space rectangular coordinate system and the three-dimensional coordinates of the vehicle door within the space rectangular coordinate system, and determine a spatial state change relationship between the vehicle surrounding obstacle and the vehicle door based on the distance.

4. The vehicle door control system of claim 3, wherein the controller is further configured to,

determining that the obstacle around the vehicle is in a dangerous space of the vehicle door when the distance is smaller than a first threshold value;

determining that the obstacle around the vehicle is in a safe space of the vehicle door when the distance is greater than a second threshold value;

and when the distance is larger than or equal to a first threshold value and smaller than or equal to a second threshold value, determining that the obstacle around the vehicle is in the critical space of the vehicle door, wherein the first threshold value is smaller than the second threshold value.

5. The vehicle door control system of claim 4, wherein the controller is further configured to,

when the obstacle around the vehicle is in the dangerous space of the vehicle door and the vehicle door is in a stop state, controlling the vehicle door to prohibit execution of the received door opening and closing command;

when the obstacle around the vehicle is in the dangerous space of the vehicle door and the vehicle door is in a moving state, controlling the vehicle door to stop moving and performing anti-collision protection;

controlling the vehicle door to execute the received door opening and closing command when the obstacle around the vehicle is in a safe space of the vehicle door and the vehicle door is in the stopped state;

When the obstacle around the vehicle is in the safety space of the vehicle door and the vehicle door is in the motion state, controlling the vehicle door to execute the received door opening and closing command;

when the obstacle around the vehicle is in the critical space of the vehicle door and the vehicle door is in a stop state, controlling the vehicle door to delay executing the received door opening and closing instruction and performing anti-collision protection;

and when the surrounding obstacle is in the critical space of the vehicle door and the vehicle door is in a moving state, controlling the vehicle door according to the movement time required by the surrounding obstacle to move to the dangerous space of the vehicle door.

6. The vehicle door control system of claim 5, wherein the controller is further configured to,

when the door is executing a door closing instruction and the movement time is less than or equal to the residual closing time of the door, controlling the door to stop moving and performing anti-collision protection;

and when the door is executing the door closing command and the movement time period is longer than the remaining closing time period of the door, controlling the door to continue executing the door closing command.

7. The vehicle door control system of any of claims 2-6, wherein the vehicle radar includes a first radar and a second radar, and the vehicle camera, the first radar, and the second radar are located in a same straight line, the controller further configured to establish the space rectangular coordinate system with the vehicle camera as an origin and the straight line as an x-axis.

8. The vehicle door control system of claim 7, wherein the three-dimensional coordinates of the obstacle surrounding the vehicle include a first x-axis component, a first y-axis component, and a z-axis component, wherein the controller is further configured to,

according to the ultrasonic information acquired by the first radar and the ultrasonic information acquired by the second radar, determining the projection length of a connecting line of the first radar and the obstacle around the vehicle on the x axis, determining the first x axis component according to the distance between the first radar and the vehicle-mounted camera and the projection length, determining the two-dimensional coordinates of the obstacle around the vehicle in a two-dimensional imaging image according to the image information, determining the first y axis component and the z axis component according to the first x axis component and the two-dimensional coordinates, and taking the first x axis component, the first y axis component and the z axis component as the three-dimensional coordinates of the obstacle around the vehicle.

9. The vehicle door control system of claim 8, wherein the two-dimensional coordinates include a second x-axis component and a second y-axis component, wherein the controller is further configured to,

determining a proportional relation between a shooting size of the vehicle-mounted camera and a size of the two-dimensional imaging image according to the first x-axis component and the second x-axis component, determining the first y-axis component according to the proportional relation and the second y-axis component, and determining the z-axis component according to the proportional relation.

10. The vehicle door control system of claim 1, further comprising a display coupled to the controller, wherein the controller is further configured to control the display to dynamically display the spatial state change relationship.

11. A method of controlling a vehicle door, the method comprising:

acquiring image information of obstacles around a vehicle, and acquiring ultrasonic information of the obstacles around the vehicle;

performing fusion processing on the image information of the surrounding obstacles of the vehicle and the ultrasonic information of the surrounding obstacles of the vehicle to obtain a space state change relation between the surrounding obstacles of the vehicle and a vehicle door;

and controlling the vehicle door according to the space state change relation.