CN115743093A

CN115743093A - Vehicle control method and device, automatic parking auxiliary controller, terminal and system

Info

Publication number: CN115743093A
Application number: CN202210916224.1A
Authority: CN
Inventors: 吴智宽; 李剑平
Original assignee: Huizhou Desay SV Automotive Co Ltd
Current assignee: Huizhou Desay SV Automotive Co Ltd
Priority date: 2022-08-01
Filing date: 2022-08-01
Publication date: 2023-03-07
Also published as: WO2024027112A1

Abstract

The invention discloses a vehicle control method, a vehicle control device, an automatic parking auxiliary controller, a terminal and a system. The vehicle control method is applied to an APA controller for assisting automatic parking, and comprises the following steps: sending images acquired by a plurality of cameras arranged on a vehicle and camera calibration data to a terminal so that the terminal can determine parking space information according to the images and the camera calibration data and construct an electronic map; receiving parking space information and electronic map information returned by a terminal; and determining a parking track according to the parking space information, the electronic map information and the radar data, and controlling the vehicle to park in the target parking space according to the parking track. According to the technical scheme, the APA controller sends the image and the camera head calibration data to the terminal, the parking space information is determined at the terminal, the electronic map is constructed, and then the terminal returns the parking space information and the electronic map information to the APA controller, so that the APA controller determines the parking track and controls the vehicle to park in the target parking space according to the parking track, and hardware resources of the vehicle are saved.

Description

Vehicle control method and device, automatic parking auxiliary controller, terminal and system

Technical Field

The embodiment of the invention relates to the technical field of vehicle control, in particular to a vehicle control method, a vehicle control device, an automatic parking auxiliary controller, a terminal and a system.

Background

With the development of vehicle control technology, the automatic parking technology is gradually integrated into more vehicle models. The automatic parking technology can acquire images collected by a radar arranged on a vehicle through a visual algorithm and perform image processing, further determine parking space information and construct an electronic map, and then determine a parking track by combining a regulation and control algorithm, further control the vehicle to park.

In the prior art, a visual algorithm of an Automatic parking technology is mostly deployed on an Automatic Park Assist (APA) of a vehicle, so that hardware devices on the vehicle are complicated. When The vision algorithm needs to be updated to The vehicle after being upgraded, whether The hardware resource of The vehicle can support The updated vision algorithm needs to be considered, and The vision algorithm needs to be updated for many times through an Over The Air (OTA) technology when being updated, so that The updating of The vision algorithm becomes complicated.

Disclosure of Invention

The invention provides a vehicle control method, a vehicle control device, an automatic parking auxiliary controller, a terminal and a system.

In a first aspect, an embodiment of the present invention provides a vehicle control method applied to an automatic parking assist APA controller, including:

sending images collected by a plurality of cameras arranged on a vehicle and camera calibration data to a terminal so that the terminal can determine parking space information according to the images and the camera calibration data and construct an electronic map;

receiving parking space information and electronic map information returned by the terminal;

determining a parking track according to the parking space information, the electronic map information and radar data, and controlling the vehicle to park in a target parking space according to the parking track.

In a second aspect, an embodiment of the present invention provides a vehicle control method, applied to a terminal, including:

receiving images and camera calibration data sent by an APA controller, wherein the images are acquired by a plurality of cameras arranged on a vehicle;

determining parking space information according to the image and the camera calibration data and constructing an electronic map;

and returning parking space information and electronic map information to the APA controller so that the APA controller determines a parking track according to the parking space information, the electronic map information and radar data and controls the vehicle to park in a target parking space according to the parking track.

In a third aspect, an embodiment of the present invention provides a vehicle control apparatus including:

the system comprises a sending module, a storage module and a display module, wherein the sending module is used for sending images acquired by a plurality of cameras arranged on a vehicle and camera calibration data to a terminal so that the terminal can determine parking space information according to the images and the camera calibration data and construct an electronic map;

the information receiving module is used for receiving parking space information and electronic map information returned by the terminal;

and the control module is used for determining a parking track according to the parking space information, the electronic map information and the radar data and controlling the vehicle to park in a target parking space according to the parking track.

In a fourth aspect, an embodiment of the present invention provides a vehicle control apparatus, including:

the automatic parking assistant APA system comprises a receiving module, a judging module and a judging module, wherein the receiving module is used for receiving images and camera calibration data sent by an automatic parking assistant APA controller, and the images are collected by a plurality of cameras arranged on a vehicle;

the construction module is used for determining parking space information according to the image and the camera calibration data and constructing an electronic map;

and the information returning module is used for returning parking space information and electronic map information to the APA controller so that the APA controller determines a parking track according to the parking space information, the electronic map information and radar data and controls the vehicle to park in a target parking space according to the parking track.

In a fifth aspect, an embodiment of the present invention provides an automatic parking assist controller, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the vehicle control method according to the first aspect.

In a sixth aspect, an embodiment of the present invention provides a terminal, including:

at least one processor; and

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the vehicle control method of the second aspect.

In a seventh aspect, an embodiment of the present invention provides a vehicle control system, including the APA controller according to the fifth aspect, and the terminal according to the sixth aspect.

According to the technical scheme of the embodiment of the invention, the APA controller sends the image and the camera calibration data to the terminal, the parking space information is determined at the terminal, the electronic map is constructed, and then the terminal returns the parking space information and the electronic map information to the APA controller, so that the APA controller determines the parking track and controls the vehicle to park in the target parking space according to the parking track, the hardware resource of the vehicle is saved, and the user experience is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a flow chart of a vehicle control method according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating a method for controlling a vehicle to park in a target parking space according to a parking trajectory according to an embodiment of the present invention;

FIG. 3 is a schematic diagram illustrating another method for controlling a vehicle to park in a target parking space according to a parking trajectory according to an embodiment of the present invention;

FIG. 4 is a flow chart of a vehicle control method according to a second embodiment of the present invention;

fig. 5 is a schematic diagram illustrating that parking space information is determined according to an image and calibration data of a camera provided by the second embodiment of the present invention;

fig. 6 is a schematic diagram of an APA controller connected to a mobile phone according to a second embodiment of the present invention;

FIG. 7 is a flowchart of another vehicle control method according to a second embodiment of the present invention;

fig. 8 is a schematic structural diagram of a vehicle control device according to a third embodiment of the invention;

fig. 9 is a schematic structural diagram of a vehicle control apparatus according to a fourth embodiment of the invention;

fig. 10 is a schematic configuration diagram of an automatic parking assist controller that implements the vehicle control method of the embodiment of the invention;

fig. 11 is a schematic configuration diagram of a terminal that implements a vehicle control method of an embodiment of the invention;

fig. 12 is a schematic configuration diagram of a vehicle control system that implements a vehicle control method of an embodiment of the invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second", etc. in the present invention are used for distinguishing similar objects, and are not necessarily used for describing a particular order or sequence. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It is understood that before the technical solutions disclosed in the embodiments of the present invention are used, the types, the usage ranges, the usage scenarios, etc. of the personal information related to the present disclosure should be informed to the user and authorized by the user in a proper manner according to the relevant laws and regulations.

Example one

Fig. 1 is a flowchart of a vehicle control method according to an embodiment of the present invention, where the present embodiment is applicable to a case where an automatic parking assist APA controller controls a vehicle to perform automatic parking, and the method may be executed by a vehicle control device, which may be implemented in software and/or hardware and integrated in the automatic parking assist controller. The automatic parking auxiliary controller can sense a parking environment through the camera and the radar, and automatically or manually set a target parking space according to the selection of a user so that the vehicle can automatically park along a parking track until the final target parking space is reached.

As shown in fig. 1, the method includes:

and S110, sending the images acquired by the plurality of cameras arranged on the vehicle and the camera calibration data to the terminal so that the terminal can determine the parking space information according to the images and the camera calibration data and construct an electronic map.

The camera may refer to a video input device. The image may be a generic term of various figures and images collected by the camera, and the image may be a photo or a video.

The number of cameras arranged on the vehicle is not limited as long as images can be acquired through the cameras. For example, the number of the cameras on the vehicle can be determined according to the actual application requirements.

The type of the camera is not limited as long as the camera can collect images, such as a wide-angle camera, a fisheye camera, a monocular camera, a binocular camera or a common visual angle camera.

The positions of the plurality of cameras arranged on the vehicle are not limited, as long as images can be acquired through the cameras, and the positions of the cameras on the vehicle can be determined according to the number of the cameras or the actual application requirements. The cameras at different positions can have different functions, for example, the camera arranged on the front windshield of the vehicle can be used for automatically detecting pedestrians, vehicles, route marks or signal lamps and the like; for another example, a camera mounted on a side of a vehicle may be used for blind spot monitoring, because the range of a vehicle rearview mirror is limited, a visual blind zone is easily formed in a place where the rearview mirror cannot be observed, the blind zone may be basically covered by mounting the camera on the side of the vehicle, when the vehicle enters the blind zone, a user may observe that the vehicle enters the blind zone through the camera on the side, and the occurrence of traffic accidents may be reduced to a certain extent.

In one embodiment, the type of the camera may be determined according to the setting position of the camera on the vehicle, for example, the camera mounted on the front windshield of the vehicle may be a monocular camera or a binocular camera, and the monocular camera or the binocular camera may acquire distance information through the acquired image, and may acquire the distance between the vehicle and a pedestrian, other vehicles, or an obstacle; for another example, the camera mounted on the side of the vehicle may be a wide-angle camera, and a user may observe the situation of the side of the vehicle through the camera by mounting a plurality of wide-angle cameras on the side of the vehicle; for another example, the camera installed at the rear of the vehicle may be a wide-angle camera or a fisheye camera, and may be used for parking assistance, so that the user may observe the situation behind the vehicle through the camera at the rear of the vehicle when parking.

In one embodiment, the number of the cameras arranged on the vehicle is at least four, the cameras are wide-angle cameras or fisheye cameras, the four cameras are respectively arranged in four directions of the vehicle, namely front, back, left and right, the images collected by the four cameras are spliced to obtain a panoramic image around the vehicle, and road route sensing or parking control and the like can be realized by combining an algorithm.

The camera calibration data may be data that reflects the relative position of the camera on the vehicle. The positions of a plurality of cameras arranged on the vehicle can be represented through the camera calibration data, and the positions of the cameras relative to the center of the vehicle can be calibrated in a coordinate system by establishing the coordinate system with the center of the vehicle as an origin.

In one embodiment, the number of the cameras and the relative positions of the cameras on the vehicle are predetermined during vehicle production, and then the calibration data of the cameras are determined to be unchanged.

In one embodiment, the number of the cameras and the relative positions of the cameras on the vehicle can be determined according to actual conditions, so as to determine the calibration data of the cameras, and when the number of the cameras or the relative positions of the cameras on the vehicle are changed, the calibration data of the cameras are changed accordingly.

The terminal may refer to an input/output device, and the type of the terminal is not limited in the present invention, and may be a notebook, a tablet computer, a laptop computer, a personal digital assistant, and other suitable computers; as another example, various forms of mobile terminals are also possible, such as personal digital processing, smart phones, wearable devices, and other similar devices.

The mode of sending the images collected by the plurality of cameras arranged on the vehicle and the camera calibration data to the terminal is not limited as long as the images collected by the plurality of cameras arranged on the vehicle and the camera calibration data can be sent to the terminal. If the wireless connection is established with the terminal through verifying the authority of the terminal, images collected by a plurality of cameras arranged on the vehicle and camera calibration data are sent to the terminal.

The method for verifying the authority of the terminal is not limited, and for example, the authority may be verified by a password. The wireless connection with The terminal may be established through wireless Communication technology such as bluetooth, WIFI (wireless fidelity), or The fourth Generation Mobile Communication technology (4 g).

In the step, the images collected by the plurality of cameras arranged on the vehicle and the camera calibration data are sent to the terminal, so that the terminal determines the parking space information according to the images and the camera calibration data and constructs the electronic map, hardware resources of the vehicle are saved, and user experience is improved.

And S120, receiving parking space information and electronic map information returned by the terminal.

The parking space information may refer to position information where the vehicle may park. The parking space information may include, but is not limited to, four corner coordinates of a parking space or a parking space line, where the four corner coordinates of the parking space may be coordinates determined by establishing a coordinate system with a central position of the vehicle as an origin, and the parking space line may refer to a line used for distinguishing the parking space from other areas except the parking space. The method includes the steps that one or more parking spaces capable of being used for parking of the vehicle can be obtained through parking space information, when the parking space information indicates that a plurality of parking spaces can be used for parking, a user can select one target parking space from the plurality of parking spaces through the parking space information to enable the vehicle to park, an APA controller of the vehicle can randomly select one target parking space from the plurality of parking spaces through the parking space information to park, and the target parking space can be the parking space where the vehicle finally parks.

The electronic map may be a map that is digitally stored and referred to by using a computer technology, and is also called a digital map. The electronic map in the invention may include, but is not limited to, obstacle information, other vehicle information and parking space information around the vehicle, wherein the obstacle information around the vehicle may include, but is not limited to, the type of obstacle around the vehicle and the position where the obstacle is located, and the other vehicle information may include, but is not limited to, the position of the other vehicle relative to the center of the vehicle. The electronic map information may refer to information on an electronic map, and the electronic map may be included in the electronic map information.

The method of receiving the parking space information and the electronic map information returned by the terminal is not limited as long as the parking space information and the electronic map information returned by the terminal can be received. For example, the parking space information and the electronic map information returned by the terminal are received in a mode that the APA controller establishes wireless connection with the terminal.

By receiving the parking space information and the electronic map information returned by the terminal, the APA controller can determine the parking track according to the parking space information, the electronic map information and the radar data.

And S130, determining a parking track according to the parking space information, the electronic map information and the radar data, and controlling the vehicle to park in the target parking space according to the parking track.

The radar data may refer to data collected by a radar provided on the vehicle. The radar may be an electronic device that detects a target using an electromagnetic wave, and the radar transmits the electromagnetic wave to irradiate the target and receives an echo of the electromagnetic wave, so that information such as a distance, a distance change rate (radial velocity), an azimuth, or an altitude from the target to an electromagnetic wave transmission point may be obtained. The relative distance, relative speed, angle or moving direction between the vehicle and other objects can be detected by providing a radar on the vehicle. The radar data may include, but is not limited to, relative distance, relative speed, angle, or direction of motion between the vehicle and other objects, etc.

The type of radar provided on the vehicle is not limited as long as radar data can be acquired by the radar. Such as a millimeter wave radar, an infrared radar, an ultrasonic radar, a laser radar, or the like. Different types of radars can have different effects when collecting radar data, for example, the detection distance of a millimeter wave radar is longer, but the obstacle identification capability is weaker than that of a laser radar; for another example, the detection distance of the laser radar is long, and meanwhile, the obstacle can be accurately positioned.

The number of the radars arranged on the vehicle is not limited, the arrangement positions of the radars on the vehicle are not limited, only the radar data can be acquired through the radars, and the number of the radars and the positions of the radars can be specifically determined according to actual application requirements.

In one embodiment, 12 radars are arranged on the vehicle, the 12 radars are uniformly distributed around the vehicle, and radar data can be acquired through the 12 radars.

The parking trajectory may refer to a trajectory that controls the vehicle to park in the target parking space. Steering wheel information, vehicle speed information, gear information, brake information or vehicle lamp information and the like of the vehicle in the process of parking in the target parking space can be determined through the parking track, and therefore the vehicle can park in the target parking space according to the parking track.

The steering wheel information may be indicative of a state in which the steering wheel should be when the vehicle is parked, such as a rotational angle of the steering wheel or a rotational direction of the steering wheel. The vehicle speed information can represent the speed of the vehicle when the vehicle is parked, for example, a preset speed can be set to enable the vehicle to keep the preset speed for parking, and the preset speed can be set according to actual needs; for example, the vehicle speed may be controlled to park the vehicle according to actual conditions, the vehicle speed may be reduced when the vehicle needs to turn, or the preset speed may be maintained when the vehicle is going straight. The gear information can represent the gear at which the vehicle should park, for example, the vehicle can park according to the set gear, and the set gear can be the gear set according to actual needs; for example, the gear can be changed to park the vehicle according to actual conditions, and the gear is a reverse gear when the vehicle needs to move backwards, or a smaller gear is selected when the vehicle runs at a lower speed. The braking information may be indicative of a state of braking of the vehicle while the vehicle is parked, such as a frequency of braking or a force of braking. The lamp information can represent the state of the lamp when the vehicle parks, such as the double-flash is started when the vehicle parks, so as to prompt the surrounding vehicle to pay attention to avoiding.

The manner of determining the parking trajectory according to the parking space information, the electronic map information, and the radar data is not limited, as long as the parking trajectory can be determined according to the parking space information, the electronic map information, and the radar data. If so, the target parking space can be determined according to the parking space information; determining the position of the target parking space relative to the vehicle and the conditions of obstacles or other vehicles around the path from the vehicle to the target parking space through the electronic map information; determining relative distance, relative speed, angle or movement direction and the like between the vehicle and other objects according to the radar data; the path of the vehicle driving into the target parking space can be planned by combining the parking space information, the electronic map information and the radar data, and the parking track can be determined.

The method for controlling the vehicle to park in the target parking space according to the parking trajectory is not limited, as long as the vehicle can be controlled to park in the target parking space according to the parking trajectory. For example, determining steering wheel information, namely the state of a steering wheel when the vehicle parks, according to the parking track; determining vehicle speed information, namely the speed of the vehicle when the vehicle parks, according to the parking track; determining gear information, namely the gear at which the vehicle should be parked, according to the parking track; determining braking information, namely the braking state of the vehicle during parking according to the parking track; or determining the lamp information according to the parking track, namely the state of the lamp when the vehicle parks; and determining the hardware which is required to be associated when the vehicle completes parking according to the parking track, establishing connection with the hardware of the vehicle associated with the parking track, and controlling the hardware of the relevant vehicle to perform corresponding operation according to the parking track so that the vehicle can park in the target parking space according to the parking track.

Fig. 2 is a schematic diagram of controlling a vehicle to park in a target parking space according to a parking trajectory according to an embodiment of the present invention, as shown in fig. 2, the vehicle is parked in the target parking space vertically, and fig. 2 includes, from top to bottom: firstly, planning ahead of a target parking space to enable a vehicle to be in front of the target parking space; secondly, planning to the area of the garage opening in the target parking space, and when an obstacle is detected, re-planning to the front of the target parking space; and thirdly, arriving at the parking point planned in the second step or meeting an obstacle for re-planning, and adjusting the posture of the vehicle forwards so that the vehicle can be parked in the target parking space at a small angle.

Fig. 3 is a schematic diagram of another embodiment of the present invention for controlling a vehicle to park in a target parking space according to a parking trajectory, as shown in fig. 3, the vehicle is parked in the target parking space horizontally, and the left to right in fig. 3 are respectively: firstly, planning a vehicle to the left front of a target parking space to enable the vehicle to be positioned in the left front of the target parking space; secondly, planning the vehicle to the right and back, and determining an angle of turning into a target parking space according to the tail position of the vehicle in the first step; thirdly, parking the vehicle into a target parking space; and fourthly, adjusting the posture of the vehicle.

According to the technical scheme of the embodiment of the invention, the image and the camera calibration data are sent to the terminal through the APA controller, the parking space information is determined at the terminal, the electronic map is constructed, and then the terminal returns the parking space information and the electronic map information to the APA controller, so that the APA controller determines the parking track through the parking space information, the electronic map information and the radar data, and controls the vehicle to park in the target parking space according to the parking track, thereby saving the hardware resource of the vehicle and improving the user experience.

Further, before controlling the vehicle to park in the target parking space according to the parking trajectory, the vehicle control method further includes:

performing an association handshake with the vehicle, the association comprising at least one of: the device comprises a vehicle body control module, an electronic parking brake system, an electronic power steering system, an electronic stability control unit and an automatic transmission control unit.

The association piece can be hardware of a vehicle related to controlling parking of the vehicle, and handshaking of the association piece is performed with the vehicle, so that the APA controller is connected with the association piece of the vehicle, the APA controller can send the parking track to the corresponding association piece, and the association piece is controlled to perform corresponding operation, so that the vehicle can park in the target parking space according to the parking track.

The association comprises at least one of: the device comprises a vehicle body control module, an electronic parking brake system, an electronic power steering system, an electronic stability control unit and an automatic transmission control unit.

The Body Control Module (BCM) may be a Module for controlling a function of a common Body, such as vehicle light Control, vehicle window Control, or central Control door lock Control. The APA controller and the vehicle body control module are used for handshaking, and double flashes can be started through the vehicle body control module in the parking process so as to prompt surrounding vehicles to avoid.

An Electronic Parking Brake (EPB) may be a system that controls Parking Brake of a vehicle through an Electronic circuit, and an APA controller may be used to handshake with the EPB, so that the vehicle may be controlled to perform Parking Brake through the EPB when there is a Parking Brake demand during Parking.

An Electronic Power Steering (EPS) system may be a Power Steering system that directly depends on a motor to provide an assist torque, and the EPS may determine a rotation direction of the motor and a magnitude of an assist current according to signals of a vehicle speed sensor and a torque sensor, so as to control Steering of a Steering wheel. The EPS can provide different power-assisted effects of the motor at different vehicle speeds, so that the vehicle is light and flexible when running in low-speed steering and is stable and reliable when running in high-speed steering. The APA controller and the electronic power steering system are used for shaking hands, and the steering wheel can be controlled to rotate through the electronic power steering system in the parking process.

An Electronic Stability Control (ESC) may refer to a unit capable of controlling the stability of a vehicle, and the ESC may Control the stability of the vehicle in the longitudinal and lateral directions by combining a plurality of sensors, an Electronic Control unit, and an actuator. The APA controller and the electronic stability control unit are used for handshaking, so that the acceleration of the vehicle can be controlled by the electronic stability control unit in the parking process, and the vehicle can be kept stable.

The automatic Transmission Control Unit (TCU) may be a Unit capable of implementing automatic Transmission of a vehicle by using a computer and a power electronic driving technology, and an APA controller is handshake with the automatic Transmission Control Unit, so that a gear state of the vehicle may be automatically changed by the automatic Transmission Control Unit during parking.

The APA controller can control the corresponding association piece to execute corresponding operation according to the parking track by performing association piece handshake with the vehicle before controlling the vehicle to park in the target parking space according to the parking track, so as to control the vehicle to park in the target parking space.

Further, before determining the parking trajectory according to the parking space information, the electronic map information, and the radar data, the vehicle control method further includes:

and determining the target parking space according to the selection operation of the user on the parking space information.

The parking space information may include one or more parking spaces, and the user may select the parking space information to determine a target parking space for final parking of the vehicle.

The user does not limit the selection operation of the parking space information, as long as the user can select the parking space information to determine the target parking space. For example, a user selects parking space information through a central control screen on an APA controller of the vehicle, the central control screen may be a display screen for displaying contents such as vehicle audio, navigation or vehicle information, interaction between the user and the vehicle may be achieved through the central control screen, different parking spaces may be displayed in the central control screen, the user selects any one of the parking spaces as a target parking space through touch operation, and the APA controller determines the target parking space according to the selection operation of the user on the parking space information.

Before the parking track is determined according to the parking space information, the electronic map information and the radar data, the target parking space is determined according to the selection operation of the user on the parking space information, when the parking space information indicates that a plurality of parking spaces exist, the target parking space can be determined through the selection operation of the user, and the target parking space can be selected to better meet the requirements of the user.

Further, the vehicle control method further includes:

and splicing the images collected by the cameras according to the camera calibration data, and displaying the spliced images in a central control screen of the vehicle.

Different images can be collected by a plurality of cameras arranged at different positions on the vehicle, for example, the image in front of the vehicle can be collected by the camera arranged in front of the vehicle, the image in side of the vehicle can be collected by the camera arranged in side of the vehicle, the image in rear of the vehicle can be collected by the camera arranged in rear of the vehicle, and the images collected by the cameras are spliced together to obtain a complete view around the vehicle.

Images acquired by the cameras are spliced according to the camera calibration data, the spliced images are displayed in a central control screen of the vehicle, the images acquired by different cameras can be spliced into a complete 360-degree all-round view field and displayed in the central control screen, a user can observe the conditions around the vehicle through the central control screen, and the parking process is safer.

Example two

Fig. 4 is a flowchart of a vehicle control method according to a second embodiment of the present invention, where this embodiment is applicable to a case where a terminal determines parking space information and constructs an electronic map by receiving information sent by an APA controller, and the method may be executed by a vehicle control device, which may be implemented in a form of software and/or hardware and integrated in the terminal. The type of terminal is not limited, as may be a notebook, tablet, laptop, personal digital assistant, and other suitable computers; as another example, various forms of mobile terminals are also possible, such as personal digital processing, smart phones, wearable devices, and other similar devices.

As shown in fig. 4, the method includes:

and S210, receiving images and camera calibration data sent by the APA controller, wherein the images are acquired by a plurality of cameras arranged on the vehicle.

The images sent by the APA controller can comprise various images and videos collected by a plurality of cameras arranged on the vehicle, and the images can be photos or videos.

The camera calibration data sent by the APA controller can represent the positions of a plurality of cameras arranged on the vehicle, and the positions of the plurality of cameras relative to the center of the vehicle can be calibrated in a coordinate system by establishing the coordinate system with the center of the vehicle as an origin.

The manner of receiving the image and the camera calibration data transmitted by the automatic parking assist APA controller is not limited, as long as the image and the camera calibration data transmitted by the automatic parking assist APA controller can be received. For example, the image and the camera calibration data sent by the APA controller are received by a mode that the terminal establishes wireless connection with the APA controller.

By receiving the images and the camera calibration data sent by the APA controller, the terminal can acquire the positions of the cameras arranged on the vehicle relative to the center of the vehicle and the images acquired by the cameras, so that the terminal can determine parking space information according to the images and the camera calibration data and construct an electronic map.

And S220, determining parking space information according to the image and the camera calibration data and constructing an electronic map.

The method for determining the parking space information according to the image and the camera calibration data is not limited, as long as the parking space information can be determined according to the image and the camera calibration data. For example, images acquired by a plurality of cameras can be spliced, and in the process of image splicing, the positions of the plurality of cameras on the vehicle relative to the center of the vehicle and the images acquired by the cameras can be determined according to camera calibration data; then, performing visual angle conversion on the spliced images to form a bird's-eye view; the method comprises the steps of carrying out binarization processing on the aerial view, namely setting the gray value of pixel points on the aerial view to be 0 or 255, namely enabling the whole aerial view to show an obvious black-white effect, greatly reducing the data volume in the aerial view, and further highlighting the outline of a parking space line, so that parking space information can be determined according to images and camera calibration data. The aerial view can be a perspective view drawn by overlooking the ground from a certain point at a high altitude by a high-viewpoint perspective method according to a perspective principle, and the geometrical characteristics of ground lines can be kept in the aerial view, so that the detection of the vehicle line is facilitated.

In an embodiment, fig. 5 is a schematic diagram of determining parking space information according to an image and calibration data of a camera according to a second embodiment of the present invention, as shown in fig. 5, the camera is a camera disposed in front of a vehicle, a bird's-eye view is formed by converting an angle of view of the image collected by the camera, and then binarization processing is performed on the bird's-eye view, so that a feature (such as a black frame shown in fig. 5) conforming to a vehicle line can be extracted. Two corner coordinates or parking space lines of a parking space in front of the vehicle can be obtained by processing the aerial view. If the parking space information is combined with images collected by cameras arranged at other positions of the vehicle, complete parking space information can be acquired.

The method of constructing the electronic map according to the image and the camera calibration data is not limited, as long as the electronic map can be constructed according to the image and the camera calibration data. For example, the positions of a plurality of cameras on the vehicle relative to the center of the vehicle and images acquired by the cameras are acquired through camera calibration data, the images acquired by the cameras on the vehicle are identified through image processing software, obstacle information and other vehicle information around the vehicle in the images can be acquired, an electronic map is constructed by combining with parking space information, and the obstacle information, other vehicle information and parking space information around the vehicle can be displayed in the electronic map. The obstacle information may include, but is not limited to, the type of obstacles around the vehicle and the location of the obstacles. Other vehicle information may include, but is not limited to, the location of other vehicles relative to the center of the vehicle.

The display mode of the obstacle information and other vehicle information around the vehicle on the electronic map is not limited, for example, the obstacle information and other vehicle information around the vehicle can be displayed in the form of characters in the electronic map; for another example, obstacle information and other vehicle information around the vehicle are displayed in the form of icons in the electronic map, and the obstacle information and other vehicle information around the vehicle can be distinguished by different icons; for another example, the outlines of the obstacles and other vehicles around the vehicle are extracted through image processing, the outlines are expressed in the form of points, a coordinate system is established with the center of the vehicle as an origin, and the obstacle information and other vehicle information around the vehicle can be displayed on the electronic map in the form of points through coordinates.

The display mode of the parking space information on the electronic map is not limited, for example, the parking space information can be displayed on the electronic map in a square frame form; for another example, a coordinate system is established with the center of the vehicle as the origin, and the parking space information is displayed in the form of coordinates.

The parking space information is determined according to the image and the camera calibration data, and the electronic map is constructed, so that the parking track can be determined conveniently through the parking space information and the electronic map.

And S230, returning the parking space information and the electronic map information to the APA controller, so that the APA controller determines a parking track according to the parking space information, the electronic map information and the radar data and controls the vehicle to park in the target parking space according to the parking track.

The electronic map information may include the electronic map constructed in step S220 according to the image and the camera calibration data. The parking space information may be the parking space information determined according to the image and the calibration data of the camera in step S220.

The parking space information returned to the APA controller may include, but is not limited to, four corner coordinates of parking spaces or a parking space line, and returning the parking space information to the APA controller may enable the APA controller to acquire one or more parking spaces that can be used for parking the vehicle. When the parking space information indicates that a plurality of parking spaces can be used for parking, the user can select one target parking space from the plurality of parking spaces through the parking space information in the APA controller to park the vehicle, or the APA controller of the vehicle can randomly select one target parking space from the plurality of parking spaces through the parking space information to park the vehicle.

The electronic map information returned to the APA controller may include, but is not limited to, obstacle information, other vehicle information, and parking space information around the vehicle, and by returning the electronic map information to the APA controller, the APA controller may be caused to acquire the position of the target parking space relative to the vehicle, and the condition of obstacles or other vehicles around the path from the vehicle to the target parking space.

According to the technical scheme of the embodiment of the invention, the image and the camera calibration data sent by the APA controller are received by the terminal, the parking space information is determined at the terminal, the electronic map is constructed and returned to the APA controller, so that the APA controller determines the parking track according to the parking space information, the electronic map information and the radar data and controls the vehicle to park in the target parking space according to the parking track, the hardware resource of the vehicle is saved, and the user experience is improved.

The invention is described below by way of example:

taking a terminal as a mobile phone as an example, a method for realizing parking is provided, in which a visual algorithm (namely, determining parking space information according to an image and camera calibration data and constructing an electronic map) is arranged in mobile phone software (Application, APP), and a rule control algorithm (namely, determining a parking track according to the parking space information, the electronic map information and radar data and controlling a vehicle to park in a target parking space according to the parking track) is arranged in an APA controller.

Fig. 6 is a schematic diagram of an APA controller connected to a mobile phone according to a second embodiment of the present invention; as shown in fig. 6, camera and radar that set up on the vehicle can be connected with the APA controller, image and radar data transmission to the APA controller that gather the camera, the camera can be four cameras that set up around the vehicle, the radar is twelve, can have the rule control algorithm in the APA controller, be used for with user interaction's state machine and associated control module, can have the vision algorithm in the cell-phone APP, the APA controller is connected through WIFI or bluetooth with cell-phone APP. The APA Controller may communicate with a body CAN through a Controller Area Network (CAN). A display of the vehicle center (i.e., the center screen) may present the video content incoming from the 360-degree look-around controller to the user for viewing.

The front camera, the rear camera, the left camera, the right camera and the left camera can acquire image information in real time. Twelve radars can detect the distance in real time. The APA controller can be in communication connection with the mobile phone APP, images and camera calibration data acquired by the camera are acquired through the APA controller, the images and the camera calibration data are sent to a visual algorithm of the mobile phone APP through a WIFI or Bluetooth communication mode, the visual algorithm is enabled to identify parking places and construct electronic map information through image processing, the mobile phone APP sends visual algorithm results (namely parking place information and electronic map information) to a regulation and control algorithm on the APA controller, and a parking track is determined through the regulation and control algorithm, so that parking is completed.

Fig. 7 is a flowchart of another vehicle control method according to the second embodiment of the present invention, as shown in fig. 7, an APA controller is connected to a mobile phone APP, the APA controller sends an image and camera calibration data to the mobile phone APP in real time, a visual algorithm of the mobile phone APP performs image processing, if a parking space is identified, parking space information and electronic map information are sent back to the APA controller, the APA controller displays a parking space map, a user can select a target parking space according to the parking space information through the parking space map at the APA controller, if the user confirms parking, the APA controller performs association handshake, and a regulation algorithm performs track calculation (i.e., determines a parking track) to control the vehicle to park.

By running the visual algorithm on the mobile phone APP, hardware resources of embedded equipment on the vehicle are saved, and the cost is reduced; meanwhile, the mobile phone updating period is short, and the opportunity of obtaining the mobile phone by a user is high after a stronger algorithm chip appears; the mobile phone resources are more abundant, the running performance is better, a better visual algorithm can be run, and visual algorithms of different grades can be run according to the mobile phone hardware resources; the mobile phone APP is stably, conveniently and quickly updated, and the risk of controller failure after the OTA updates the vision algorithm of the embedded device is reduced; the input parameters of the visual algorithm can be flexibly adjusted to adapt to various vehicles.

EXAMPLE III

Fig. 8 is a schematic structural diagram of a vehicle control device according to a third embodiment of the present invention, which is applicable to a case where an automatic parking assist APA controller controls a vehicle to perform automatic parking. As shown in fig. 8, the specific structure of the apparatus includes:

the sending module 31 is configured to send images acquired by a plurality of cameras arranged on a vehicle and camera calibration data to the terminal, so that the terminal determines parking space information according to the images and the camera calibration data and constructs an electronic map;

the information receiving module 32 is used for receiving the parking space information and the electronic map information returned by the terminal;

and the control module 33 is configured to determine a parking track according to the parking space information, the electronic map information, and the radar data, and control the vehicle to park in the target parking space according to the parking track.

The vehicle control device provided in this embodiment first sends images collected by a plurality of cameras arranged on a vehicle and camera calibration data to a terminal through a sending module 31, so that the terminal determines parking space information according to the images and the camera calibration data and constructs an electronic map; then, the parking space information and the electronic map information returned by the terminal are received through the information receiving module 32; and finally, determining a parking track through the control module 33 according to the parking space information, the electronic map information and the radar data, and controlling the vehicle to park in the target parking space according to the parking track.

Further, before controlling the vehicle to park in the target parking space according to the parking trajectory, the apparatus further includes:

the association piece handshake module is used for performing association piece handshake with the vehicle, and the association piece comprises at least one of the following components: the device comprises a vehicle body control module, an electronic parking brake system, an electronic power steering system, an electronic stability control unit and an automatic transmission control unit.

Further, before determining the parking trajectory according to the parking space information, the electronic map information, and the radar data, the apparatus further includes:

and the target parking space determining module is used for determining the target parking space according to the selection operation of the user on the parking space information.

Further, the apparatus further comprises:

and the spliced image display module is used for splicing the images acquired by the cameras according to the camera calibration data and displaying the spliced images in a central control screen of the vehicle.

The vehicle control device provided by the embodiment of the invention can execute the vehicle control method provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

Example four

Fig. 9 is a schematic structural diagram of a vehicle control device according to a fourth embodiment of the present invention, which is applicable to a case where a terminal determines parking space information and constructs an electronic map by receiving information sent by an APA controller. As shown in fig. 9, the specific structure of the apparatus includes:

the receiving module 41 is configured to receive an image and camera calibration data sent by the APA controller, where the image is acquired by multiple cameras disposed on a vehicle;

the construction module 42 is configured to determine parking space information according to the image and the camera calibration data and construct an electronic map;

and an information returning module 43, configured to return the parking space information and the electronic map information to the APA controller, so that the APA controller determines a parking track according to the parking space information, the electronic map information, and the radar data, and controls the vehicle to park in the target parking space according to the parking track.

The vehicle control device provided in this embodiment first receives an image and camera calibration data sent by an automatic parking assist APA controller through a receiving module 41, where the image is acquired by a plurality of cameras disposed on a vehicle; then, the construction module 42 determines parking space information according to the images and the camera calibration data and constructs an electronic map; and finally, returning parking space information and electronic map information to the APA controller through the information returning module 43, so that the APA controller determines a parking track according to the parking space information, the electronic map information and radar data and controls the vehicle to park in the target parking space according to the parking track.

The vehicle control device provided by the embodiment of the invention can execute the vehicle control method provided by the second embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE five

Fig. 10 is a schematic configuration diagram of an automatic parking assist controller that implements the vehicle control method of the embodiment of the invention. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 10, the automatic parking assist controller 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, where the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the automatic parking assist controller 10 may also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

A plurality of components in the automatic parking assist controller 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the automatic parking assist controller 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as a vehicle control method.

In some embodiments, the vehicle control method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed on the automatic parking aid controller 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the vehicle control method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the vehicle control method by any other suitable means (e.g., by means of firmware).

Fig. 11 is a schematic configuration diagram of a terminal that implements the vehicle control method of the embodiment of the invention. A terminal is intended to mean a notebook, tablet, laptop, personal digital assistant, and other suitable computers, or various forms of mobile terminals such as personal digital processing, smart phones, wearable devices, and other similar devices.

As shown in fig. 11, the terminal 20 includes at least one processor 21, and a memory communicatively connected to the at least one processor 21, such as a Read Only Memory (ROM) 22, a Random Access Memory (RAM) 23, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 21 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 22 or the computer program loaded from the storage unit 28 into the Random Access Memory (RAM) 23. In the RAM 23, various programs and data necessary for the operation of the terminal 20 can also be stored. The processor 21, the ROM 22, and the RAM 23 are connected to each other via a bus 24. An input/output (I/O) interface 25 is also connected to bus 24.

A number of components in the terminal 20 are connected to the I/O interface 25, including: an input unit 26 such as a keyboard, a mouse, or the like; an output unit 27 such as various types of displays, speakers, and the like; a storage unit 28, such as a magnetic disk, optical disk, or the like; and a communication unit 29 such as a network card, modem, wireless communication transceiver, etc. The communication unit 29 allows the terminal 20 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The processor 21 may be any of various general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 21 performs the various methods and processes described above, such as the vehicle control method.

In some embodiments, the vehicle control method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and/or installed on the terminal 20 via the ROM 22 and/or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps of the vehicle control method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the vehicle control method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described herein may be implemented on an automated parking assist controller or terminal having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the automatic parking assist controller or terminal. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

Fig. 12 is a schematic configuration diagram of a vehicle control system that implements a vehicle control method of an embodiment of the invention. As shown in fig. 12, the vehicle control system 50 includes an APA controller 51 and a terminal 52. The APA controller 51 and the terminal 52 may communicate by establishing a wireless connection, which is not limited, for example, the wireless connection may be established by using a wireless communication technology such as bluetooth, a wireless network communication technology WIFI, or a fourth generation mobile communication technology.

The APA controller 51 and the terminal 52 are in wireless connection, the APA controller 51 can send images collected by a plurality of cameras arranged on a vehicle and camera calibration data to the terminal 52, the terminal 52 determines parking space information and constructs an electronic map by receiving the images sent by the APA controller 51 and the camera calibration data, and then the parking space information and the electronic map information are returned to the APA controller 51, so that the APA controller 51 determines a parking track according to the parking space information, the electronic map information and radar data, and controls the vehicle to park in a target parking space according to the parking track.

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle control method, applied to an automatic parking assist APA controller, comprising:

2. The method of claim 1, further comprising, prior to controlling the vehicle to park in a target parking space according to the parking trajectory:

3. The method of claim 1, further comprising, prior to determining a parking trajectory from the parking space information, the electronic map information, and radar data:

4. The method of claim 1, further comprising:

and splicing the images acquired by the cameras according to the camera calibration data, and displaying the spliced images in a central control screen of the vehicle.

5. A vehicle control method is applied to a terminal, and the method comprises the following steps:

receiving an image and camera calibration data sent by an APA controller, wherein the image is acquired by a plurality of cameras arranged on a vehicle;

6. A vehicle control apparatus, characterized by comprising:

7. A vehicle control apparatus, characterized by comprising:

8. An automatic parking assist controller, comprising:

at least one processor; and

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the vehicle control method of any one of claims 1-4.

9. A terminal, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the vehicle control method of claim 5.

10. A vehicle control system comprising the APA controller of claim 8 and the terminal of claim 9.