RU2800036C1 - Method for controlling automated parking system and devices for its implementation - Google Patents

Abstract

FIELD: automotive industry.

SUBSTANCE: group of inventions is related in particular to a method for controlling an automated parking system and a device for its implementation. The device includes the first data acquisition module used to collect graphical and radar data around the vehicle using sensors, a neural network module used to input graphical and radar data into a given convolution neural network model and receiving from it parking place information, a "Park" action selection module used to select information about the final parking place from the general flow of information about the parking place in accordance with the received command to select a parking place, a trajectory selection module, a measurement module, and a functional category determination module.

EFFECT: possibility of early search near the car for a parking place is achieved, which increases the comfort of the user while driving.

14 cl, 17 dwg

Description

The content of claims for the right of priority in relation to the invention under the name "Method of controlling the automated parking system and a device for its implementation", an application for which was filed with the PRC Patent Office on November 29, 2019 under the number 201911206068.4, is given in this patent application in full by citation.

Technical area

The present invention relates to the automotive industry, in particular, to a method for controlling an automated parking system and devices for its implementation.

Prior Art

Parking a car in city parking lots presents considerable difficulties for people who do not have sufficient driving skills. However, with the development of automotive electronics, cars are gradually being equipped with more and more automated systems, including automated parking systems have become an important tool for eliminating difficulties when parking cars.

Currently, most automated parking systems work thanks to radar sensors and / or image sensors. As an integral part of automated parking systems, radar sensors serve mainly to collect data on the distance from the car to obstacles present nearby, and image sensors - to collect various kinds of graphic information. As a rule, after the user turns on the automated parking system, the car begins to collect data about the environment using radar sensors and image sensors, determines the exact location of parking spaces near the car and generates a parking path along which it will enter the parking space. After the user confirms the intention to park, the car starts to automatically enter the parking space along the generated parking path.

However, the automated parking systems that exist today have many disadvantages. First, the car starts looking for a parking space only after the user selects and activates the automated parking system, as a result of which they can easily miss free parking spaces. Secondly, the car can only identify common types of parking spaces with clear markings and cannot identify parking spaces in heterogeneous situations, which results in its low parking spot recognition rate. Thirdly, the vehicle may provide the user with only one parking space to choose from, and may not be able to satisfy the user's need to search for various types of parking spaces. Thus, the automated parking systems available today are imperfect and may be the cause of reduced driver comfort while driving.

Content of the invention

In view of the foregoing, the aim of the present invention is to present a new method of controlling an automated parking system and devices for its implementation, which will improve the comfort for drivers while driving by eliminating the shortcomings that exist in existing technologies due to the imperfection of automated parking systems, such as the risk of passing free parking spaces, a low recognition rate for parking spaces, and the inability to meet the needs of drivers in finding various types of parking spaces.

In order to achieve the above goal, the technical solution that is the subject of this disclosure is implemented as follows:

A method of control by an automated parking system, applicable to vehicles equipped with sensors, and including:

Graphical and radar data collected by said sensors around said vehicle moving at a speed that is within a predetermined interval, and the time during which the vehicle is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time;

Information about at least one parking space, obtained at the output of a given model of a convolutional neural network, into which said graphic and radar data are entered, and the training of the said convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types;

If the current gear the vehicle is in is forward or idle, then the final parking space information is selected from the total flow of at least one parking space information in accordance with the parking space selection command received;

Based on said information about the final parking location, a vehicle parking path is generated, along which said vehicle is automatically parked.

Further, said sensors mean a plurality of image sensors and a plurality of radar sensors, and said vehicles are equipped with electronic control units, and before graphics and radar data are collected by said sensors around said vehicles under conditions when said vehicles move at a speed whose value is within a predetermined interval, and the time during which the vehicle travels at a speed whose value is within a predetermined interval is greater than or equal to a predetermined time, this method is also characterized as follows:

When starting the said vehicle, the operating state of said image sensors, radar sensors and electronic control unit is determined, and a distinction is made between good and emergency states;

Based on said operating states, the functional category of said vehicle is determined.

Further, the functional category of said vehicle may be any of four (normal operation category, first degree accident category, second degree accident category, and third degree accident category), and said step of determining the functional category of said vehicle based on said operating states is characterized as follows:

If the working condition of said image sensors, radar sensors and electronic control unit is good, then said functional category of the vehicle is defined as said category of normal operation;

If the operating state of at least one of said radar sensors and/or the electronic control unit is emergency, then said functional category of the vehicle is defined as said emergency category of the first degree;

If the operating state of all said image sensors is an emergency, then said functional category of the vehicle is defined as said emergency category of the second degree;

If the operating state of some of said image sensors is in an emergency, then said functional category of the vehicle is defined as said emergency category of the third degree.

Next, said ECU is used to control the automatic parking process of said vehicle, and after the step of determining the functional category of said vehicle based on said operating states is carried out:

If the functional category of said vehicle is determined to be said emergency category of the first degree, then the system issues a command to terminate the functioning of said electronic control unit;

If the functional category of said vehicle is determined to be said emergency category of the second degree, then the system issues a command to terminate the operation of said image sensors;

If the functional category of said vehicle is determined to be said emergency category of the third degree, then the system commands to stop the operation of those image sensors whose said operating state is determined to be said emergency state;

Further, the information about said parking space includes one or more of any of the following types of information: information about the presence of traffic cones in the parking space, information about the presence of signs prohibiting parking in the parking space, information about the presence of a parking barrier in the parking space, information about the presence of markings in the parking space, and information about the presence of obstacles in the parking space.

Further, the step in which said graphic and radar data is input to a given CNN model to output information about at least one parking space to said CNN model includes the following steps:

Determining by said convolutional neural network model said information about the parking space, including its category, based on said graphical and radar data entered into said convolutional neural network model;

Obtaining at the output of the mentioned model of the convolutional neural network the mentioned information about the parking space, including information about the category of the parking space.

Further, after the stage is completed, at which the mentioned model of the convolutional neural network, based on the mentioned graphic and radar data entered into it, determines information about the parking space, including its category:

If said parking space is a corner one, then said CNN model determines for said corner parking space a parking direction based on said graphical and radar data;

Said parking space information output from said convolutional neural network model includes, but is not limited to, parking direction information for said corner parking space.

Next, the step of selecting the final parking space information from the information of at least one said parking space according to the received parking space selection command, if the current gear in which said vehicle is running is forward or idle, includes the following steps:

If, at the time the parking space search signal is received, the current gear in which said vehicle is running is said forward gear or said idle gear, a parking control interface is displayed containing information about at least one of said parking spaces;

Upon receipt of a command to select a parking space corresponding to said parking control interface, information about the final parking space is selected from information about at least one said parking space in accordance with the received command to select a parking space.

Further, after the step is completed, at which, based on the mentioned information about the final parking place, a trajectory for car parking is generated, along which the mentioned car is automatically parked:

If at the time of the arrival of the signal to search for a parking space, the current transmission of said vehicle is a parking gear, then data about the environment around said vehicle is collected using said sensors;

Based on said environmental data, at least one direction of exit of said vehicle from the parking space is determined;

The vehicle exit control interface is displayed, in which at least one of said parking exit directions is indicated;

When a command is received to leave the parking place, on its basis, the final direction is selected from the mentioned exit directions;

On the basis of said information about the final direction of the exit from the parking lot, a trajectory is generated for the exit of the car from the parking space, along which the said car automatically leaves the parking place.

Further, after the step is completed, at which, based on the mentioned information about the final parking place, a trajectory for car parking is generated, along which the mentioned car is automatically parked:

If an abnormal state of said vehicle is detected, the system issues a command to temporarily stop the automatic parking process;

If it is confirmed that said abnormal state of said vehicle has been eliminated, the system commands to continue the automatic parking process.

Further, the vehicle is considered to be in the aforementioned anomalous condition if at least one of the following circumstances occurs:

Said vehicle is in the state where the driver is not wearing a seat belt;

Said vehicle is in a state where its doors are not closed;

An obstacle appears on the trajectory for parking the mentioned car.

The mentioned device for implementing the control method by the automated parking system includes:

The first data acquisition module used to collect graphic and radar data around the mentioned vehicle moving at a speed whose value is within a given interval, and the time during which the vehicle moves at a speed whose value is within a given interval is greater than or equal to a given time.

A neural network module used to input said graphical and said radar data into a given CNN model and obtain, at the output of said CNN model, information about at least one parking space, wherein said CNN model is trained on the basis of training graphics and radar data collected from a large number of parking spaces of various types;

A "Park" action selection module used to select information about the final parking location from the general flow of information about at least one said parking space in accordance with the incoming command to select a parking space, if the current gear in which said vehicle is driving is forward or idle;

A path selection module used to generate a path for parking a vehicle along which said vehicle is automatically parked based on said final parking location information.

Further, said device includes, but is not limited to:

A measurement module used to determine the operating states of said image sensors, radar sensors and an electronic control unit when starting said vehicle, distinguishing between good and fault states;

A functional category determination module used to determine the functional category of said vehicle based on said operating states.

Further, said functional category definition module includes:

The first confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said normal operation category if the operating state of said image sensors, radar sensors and electronic control unit is good;

A second confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said emergency category of the first degree, if the operating state of at least one of said radar sensors and/or electronic control unit is emergency;

The third confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said emergency category of the second degree, if the operating state of all said image sensors is emergency;

The fourth confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said emergency category of the third degree, if the operating state of some of said image sensors is emergency.

Further, said device includes:

The first control module, the role of which is to issue a command to stop the operation of said electronic control unit in the event that the functional category of said vehicle is determined to be said emergency category of the first degree;

The second control module, the role of which is to issue a command to stop the operation of said image sensors in the event that the functional category of said vehicle is determined to be said emergency category of the second degree;

The third control module, the role of which is to issue a command to stop the operation of those image sensors, the said operating state of which is defined as the mentioned emergency condition, in case the functional category of the said vehicle is defined as the mentioned emergency category of the third degree.

Further, the information about said parking space includes one or more of any of the following types of information: information about the presence of traffic cones in the parking space, information about the presence of signs prohibiting parking in the parking space, information about the presence of a parking barrier in the parking space, information about the presence of markings in the parking space, and information about the presence of obstacles in the parking space.

Further, the mentioned neural network module includes:

An input submodule used to input said graphical and said radar data into the CNN model and obtain by said CNN model information about said parking space, including its category;

The first output submodule used to output said convolutional neural network model information about said parking space, including its category;

Further, the mentioned neural network module includes:

A parking direction determination sub-module in a corner parking space, used to determine by said CNN model, with respect to said corner parking space, a parking direction based on said graphic and radar data, if said parking space is a corner one;

Further, said "Park" action selection module includes:

A display sub-module used to display a parking management interface containing information about at least one of said parking spaces, if at the time of the signal to search for a parking space, the current gear in which said vehicle is running is said forward or said idle gear;

A selection submodule used to select information about the final parking space from the general flow of information about at least one said parking space when a parking space selection command corresponding to said parking control interface is received.

Further, the device includes, among other things:

The second data acquisition module used to collect data about the environment around the said vehicle using said sensors, if at the time of the signal to search for a parking space, the current gear of the said vehicle is parking;

A confirmation module used to confirm at least one driving direction of said vehicle to parking spaces based on the environment data;

A display module used to display a vehicle exit control interface comprising at least one of said exit directions;

The module for selecting the action "Leave the parking spot", used to select from the mentioned directions of exit from the parking lot the final direction of exit in accordance with the command to leave the parking lot;

A parking exit trajectory determination module used to generate a trajectory for a vehicle to exit a parking space, along which said vehicle, based on said information about the final direction of the exit, automatically exits the parking space.

Further, said device includes but is not limited to:

A parking abort module used to issue a command to temporarily abort the automatic parking process in case an abnormal state of said vehicle is detected;

A parking resuming module used to issue a command to continue the automatic parking process of said vehicle in case of confirmation that said abnormal state of said vehicle has been eliminated.

Further, the vehicle is considered to be in the aforementioned anomalous condition if at least one of the following circumstances occurs:

Said vehicle is in the state where the driver is not wearing a seat belt;

Said vehicle is in a state where its doors are not closed;

An obstacle appears on the trajectory for parking the mentioned car.

Compared with existing technologies today, the method of controlling the automated parking system and the devices for its implementation, presented by the present disclosure, have the following advantages:

The method of controlling the automated parking system and devices for its implementation, presented in the implementation example of the present disclosure, include: graphic and radar data collected by sensors around a car moving at a speed that is within a given interval, and the time during which the car is moving at a speed that is within a given interval is greater than or equal to a given time; information about at least one parking space obtained from the output of a given model of a convolutional neural network, into which graphic and radar data are entered, and the training of the convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types; final parking space information selected from the at least one parking space information stream according to the parking space selection command received, if the current gear the vehicle is in is forward or idle; a trajectory for automatic parking of the car, generated based on the information about the final parking place. The present disclosure provides for the possibility of early search near the car of a parking space by measuring the speed of the car, identifying a parking space for parking a car in heterogeneous situations by applying a convolutional neural network model, in addition, it is possible to provide information about a plurality of free parking spaces, which allows satisfying the user's needs in searching for parking spaces of various types, optimizing the automated parking system itself and improving user comfort while driving.

The foregoing is only a brief overview of the technical solutions offered by this disclosure. In order to more widely disclose the essence of the technical methods presented by this disclosure, and in accordance with the instructions to implement it, as well as to explain and make more understandable features, advantages and one of the above and other purposes of this disclosure, a specific example of its implementation is given below.

Explanations for illustrations

In order to clarify the technical solutions offered by an example implementation of this disclosure or existing technologies, the following is a summary of the illustrations used to describe an example implementation of this disclosure or existing technologies, and it is obvious that the illustrations described below refer to an example implementation of the present disclosure and can serve as a basis for ordinary technical workers in this field to obtain other illustrations without creative work.

The illustrations that form part of this disclosure are used to facilitate understanding of the essence of the disclosure presented in the application, and the illustrated implementation example and textual explanations thereto serve to interpret this disclosure and do not impose unnecessary restrictions on it. The following are explanations of the accompanying illustrations:

in FIG. 1 is a flow diagram of the method of controlling the automated parking system mentioned in the exemplary implementation of the present disclosure;

in FIG. 2 is a diagram of the basic layout of the automated parking system mentioned in the exemplary implementation of the present disclosure in a vehicle equipped with such a system;

in FIG. 3 shows a parking space search interface in the general car parking management interface mentioned in the implementation example of the present disclosure;

in FIG. 4 shows a parking space display interface in the general car parking control interface mentioned in the implementation example of the present disclosure;

in FIG. 5 is a flow diagram of another method of controlling the automated parking system mentioned in the exemplary implementation of the present disclosure;

in FIG. 6 is a schematic representation of the three-space parking classifier mentioned in the exemplary implementation of the present disclosure;

in FIG. 7 is a flow diagram of the automated parking process referred to in an embodiment of the present disclosure;

in FIG. 8 is a schematic representation of the abnormal state vehicle information mentioned in the exemplary implementation of the present disclosure;

in FIG. 9 is a schematic representation of the options mentioned in the exemplary implementation of the present disclosure, displayed after the parking process is suspended;

in FIG. 10 is a diagram of the automatic parking suspension process mentioned in the exemplary embodiment of the present disclosure;

in FIG. 11 is a flow diagram of another method of controlling the automated parking system mentioned in an embodiment of the present disclosure;

in FIG. 12 is a flow diagram of another method of controlling the automated parking system mentioned in an embodiment of the present disclosure;

in FIG. 13 is a diagram of a direction selection for exiting a parking space, the function of which is provided by the automated parking system mentioned in the exemplary implementation of the present disclosure;

in FIG. 14 is a diagram of another type of automated parking process mentioned in the exemplary implementation of the present disclosure;

in FIG. 15 is a structural drawing of the vehicle mentioned in the exemplary embodiment of the present disclosure, equipped with an automated parking system;

in FIG. 16 schematically depicts the structure of the computer processing equipment used to implement the method presented in the present disclosure; And

in FIG. 17 is a schematic representation of a memory module used to store or carry the program codes necessary to implement the method of the present disclosure.

A specific example of the implementation of the invention

In order to disclose the essence of the above problems, technical solutions and advantages of the present disclosure, the following is a full and detailed description of the technical solutions applicable to the exemplary embodiments of the invention, using illustrations in the form of drawings. However, the following example implementation of the disclosure covers only part, and not the entire scope of its application. All other ways of implementing the disclosure, obtained by ordinary technical workers in this field on the basis of the example given in this application without creative work, are objects of patent rights and are subject to protection within the framework of this application.

It should be noted that in the absence of contradictions, the embodiment of the disclosure presented in this application and its characteristic features can be mutually combined.

Below, on an example implementation with illustrations in the form of the accompanying drawings, a detailed description of the present disclosure is given.

On FIG. 1 is a flow diagram of the method of controlling the automated parking system mentioned in the exemplary implementation of the present disclosure.

Step 101: collecting by means of sensors graphic and radar data around the mentioned vehicle moving at a speed, the value of which is within a predetermined interval, and the time during which the vehicle is moving at a speed, the value of which is within a predetermined interval, is greater than or equal to a predetermined time.

Modern automobiles are equipped with numerous electronic systems that enrich the functionality of automobiles and make it possible to increase their practicality, moreover, thanks to the interaction of various types of electronic systems with each other, an increase in the level of automobile automation is possible.

As shown in the exemplary implementation of the present disclosure, a vehicle is equipped with numerous electronic systems and sensors that are part of the basic layout of an automated parking system in a vehicle equipped with such a system. A diagram of such a basic arrangement is shown in Fig. 2, which illustrates:

Radar sensors designed to collect radar data around the vehicle;

Image sensors designed to collect image data around the car;

Seat belt sensors, designed to collect information about whether the seat belts are currently fastened;

Display unit used to display vehicle information and receive user commands, including vehicle monitor, center display, etc.;

An Electronic Stability Program (ESP) system designed to control the stability of the vehicle body, actuating the brake mechanism or influencing the output torque and interacting with the engine control module (Engine Control Module, ECM) or vehicle control unit (Vehicle Control Unit, VCU);

Electric Power Steering (EPS) system designed to actuate the steering wheel and report data on the angle of its rotation;

Transmission Control Unit (TCU), designed to report data on the current gear and shift gears to the final ones;

Body Control Module (BCM), designed to report data on the status of car doors and control car door locks;

Balise Transmission Module (BTM), designed to receive and send data to terminal devices;

Terminal devices include electronic keys, mobile terminals, or other electronic equipment connected to the same network as the vehicle;

An Electronic Control Unit (ECU) designed to receive and process data collected by sensors, as well as to process and report data on the inputs and outputs of various electronic systems.

At this step, it is possible to control the vehicle speed by ESP. If the vehicle is traveling at a speed whose value is within a given interval, and the time during which it moves at a speed whose value is within a given interval is greater than or equal to a given time, then the image sensors with which it is equipped collect graphic data around it, and the radar sensors collect radar data.

Specifically, in most cases when the vehicle is about to stop, its speed is kept within relatively low ranges, for example between 0 and 25 km/h. At the same time, in order to avoid starting the car, an additional condition is set for the time during which the car moves at a speed whose value is within the specified interval. For example, if the vehicle speed is maintained between 0 and 25 km/h, and the time during which this speed is maintained is 10 seconds, then the ECU can, based on the data sent by the ESP about the vehicle speed, turn on the image sensors and radar sensors to begin collecting graphic and radar data around the vehicle.

For example, when a user enters a parking lot to find a place to park a car and starts to slow down to 0-25 km/h, the time during which this speed is maintained exceeds 10 seconds, then the image sensors and radar sensors begin to collect graphic and radar data around the car and send the collected data to the electronic control unit.

According to this example implementation of the present disclosure, if the result of measuring the speed of the car shows that it meets the specified conditions, then the car begins to independently collect graphic and radar data around itself. The electronic control unit proactively receives the graphic and radar data collected by the image sensors and the radar sensors, respectively, before the user starts the automated parking system, and subsequently analyzes the parking spaces from this data.

Step 102: inputting said graphical and radar data into a given model of a convolutional neural network and obtaining information about at least one parking space as an output of said model, wherein training of said convolutional neural network model is carried out on the basis of training graphical and radar data collected from a large number of parking spaces of various types.

According to this embodiment of the present disclosure, a convolutional neural network is a feed-forward neural network containing convolution calculations and having a deep architecture, as well as one of the representative deep learning algorithms. Convolutional neural networks have high fault tolerance and excellent parallel processing and self-learning capabilities, they can solve problems resulting from the heterogeneity of environmental information, fuzzy background knowledge, and unclear inference norms, while it is acceptable if the samples have significant defects or distortions, in addition, convolutional neural networks are characterized by high responsiveness, good self-adaptation ability and high resolution, so they can achieve high accuracy and speed in identifying parking spaces under conditions where The parking lot has various types of parking spaces.

To build a model of a convolutional neural model, you can pre-configure various parameters in it, such as the parameters of the hidden and output layers of the network. The process of building a convolutional neural network consists of the following steps: firstly, graphical and radar training data collected from a large number of parking spaces of various types are entered into pre-configured hidden layers of a convolutional neural network, where graphical and radar recognition features are extracted from them; secondly, the extracted features of graphical and radar recognition are classified by means of the output layers of the convolutional neural network; thirdly, if the resulting classification information about the parking space matches the information about the parking space corresponding to all the graphic and radar training data, then the training process of the convolutional neural network is considered completed and resulted in a trained model of the convolutional neural network.

Further, if the resulting classification information about the parking space does not match the information about the parking space corresponding to all the graphical and radar training data, then by adjusting the presets, the parameters of the hidden and output layers of the network can be changed so that the resulting classification information about the parking space coincides with the information about the parking space corresponding to all the graphical and radar training data, and the process of training the convolutional neural network can be considered completed and resulted in a trained model of the convolutional neural network.

At this step, it is possible to enter graphic and radar data into the tuned CNN model in the electronic control unit, if such a model is trained and graphic and radar data are received in the electronic control unit, so that the CNN model outputs information about the parking space.

Further, valid and invalid parking space information is distinguished. Real information is understood as information about a parking space in which a car can be parked, while the real information about a parking space output by a convolutional neural network model can be stored by an electronic control unit in a local storage as information about a considered parking space. Invalid refers to information about a parking space in which it is impossible to park a car, while the invalid information about the parking space output by the CNN model can be deleted by the electronic control unit.

For example, if the user intends to park the car in a parking lot, then while the car is still moving, the sensors are already beginning to collect graphic and radar data about a large number of parking spaces and transmit them to the electronic control unit. Some of these parking spaces are equipped with parking barriers, some are occupied by other parked cars, but there are also free spaces where the user can park his car. The electronic control unit can input graphical and radar data into a predetermined convolutional neural network model, which subsequently outputs information about the parking space, including if there is a parking barrier in the parking space or another car is parked, then the information about such a parking space is invalid, and if the parking space is empty, then the information about it is valid. Valid information about parking spaces is stored by the electronic control unit, and invalid information is deleted by it.

The optimal solution proposed by the present disclosure is that the electronic control unit can not only store information about a large number of parking spaces, but also estimate the distance based on the stored information. If the electronic control unit detects that the actual distance between the car and the stored information about the parking space exceeds the specified value, it deletes such information from its memory so as not to overload it, and also not to provide the user with information about parking spaces that are far from the location of the car, and not cause him inconvenience in this regard.

According to this embodiment of the present disclosure, the ECU can output information about a parking space by inputting graphic and radar data using a predetermined CNN model, and furthermore, with the CNN model, the ECU can process graphic and radar data under a variety of different and non-uniform conditions in order to accurately recognize truly free parking spaces and improve user driving comfort.

Step 103: Selecting from the general flow of information about at least one parking space, information about the final parking space according to the received command to select a parking space, if the current gear in which the car is moving is forward or idle.

At this step, it is possible to determine and transmit to the TCU information about the current gear in which the vehicle is traveling to the electronic control unit. If the current gear of the car is forward or idle, then the electronic control unit, in accordance with the received command to select a parking space, selects information about the final parking space from the general information flow about at least one parking space.

Specifically, a user intending to automatically park a car can open an automatic car parking control interface. After receiving such a signal, the electronic control unit through the TCU receives information about the gear in which the car is currently moving, and if the current car gear is forward or idle, the automatic parking control interface is replaced by the parking control interface. If parking space information is already stored in the ECU before the display unit displays the parking management interface, this information is displayed in the parking management interface. If there is no parking space information in the ECU before the display unit displays the parking management interface, the parking management interface reminds the user to keep driving the car and continuously collect graphic and radar data around the car with sensors until the parking space information is stored in the ECU and the information is displayed in the parking management interface. On FIG. 3 shows a parking space search interface in the general car parking control interface mentioned in the implementation example of the present disclosure.

Further, the parking space driving interface can simultaneously display information about a plurality of parking spaces, one of which can be selected by the user according to his actual needs. After the parking space is selected by the user, i.e. after the user gives a command to park the car, the electronic control unit selects information about the final parking space from such a command.

For example, in FIG. 4 shows a parking space display interface in the general car parking management interface mentioned in the implementation example of the present disclosure. As seen in FIG. 4, when the car detects 3 free parking spaces nearby, the car monitor displays these 3 free spaces for the user to select from. These parking spaces are marked with the letters A, B and C, respectively.

According to this example of the implementation of this disclosure, the electronic control unit can store and display information about many parking spaces for the user when the current gear on which the car is moving is the front or idle, then, on command from the user about parking of the car, it selects the final place for parking from the general flow of information about at least one parking place, and when the user needs to choose the place of parking in accordance Ki, offers the user to choose from information about many parking spaces and thereby increases the user's comfort during driving.

Step 104: Generate a car parking path that said car automatically parks based on said final parking location information.

At this step, it is possible for the electronic control unit to generate a trajectory for parking the car based on information about the final parking space, while the electronic control unit, interacting with the ESP, EPS and TCU in controlling the direction of movement, as well as the processes of gear shifting, braking and stopping, automatically starts the car to the final parking space along the generated trajectory.

Specifically, information about a specific parking space may contain information about the location of this place, and the electronic control unit can compare the actual location of the car with the location of the parking space, calculate the relative distance between the car and the parking space, and, based on the result of the calculations, compile the necessary coordinate system. Such a coordinate system will allow the electronic control unit to determine the place from the coordinates, the turning radius and the angle enclosed between the car and the parking space in order to calculate the trajectory of arrival at the parking space. When the car starts to park automatically, the ESP can continuously monitor the speed and braking processes of the vehicle by the commands given by the ECU, the EPS can continuously control the direction of the vehicle by commands given by the ECU, and the TCU can continuously control the gear shifting by commands given by the ECU until the car is fully driven into the final parking space and the automatic parking process is completed.

Based on the foregoing, the method of controlling an automated parking system presented by the present disclosure includes: graphic and radar data collected by sensors around a car moving at a speed that is within a predetermined interval, and the time during which the car is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; information about at least one parking space obtained from the output of a given model of a convolutional neural network, into which graphic and radar data are entered, and the training of the convolutional neural network model is based on training graphic and radar data collected from a large number of parking spaces of various types; final parking space information selected from the at least one parking space information stream according to the parking space selection command received, if the current gear in which the vehicle is running is forward or idle; a trajectory for automatically parking a car generated based on the information about the final parking location. The present disclosure provides for the possibility of early search near the car of a parking space by measuring the speed of the car, identifying a parking space for parking a car in heterogeneous situations by applying a convolutional neural network model, in addition, it is possible to provide information about a plurality of free parking spaces, which allows satisfying the user's needs in searching for parking spaces of various types, optimizing the automated parking system itself, and improving user comfort while driving.

On FIG. 5 is a flow diagram of another method of controlling the automated parking system mentioned in the exemplary implementation of the present disclosure.

Step 201: Collecting by means of sensors graphic and radar data around said vehicle moving at a speed whose value is within a given interval, and the time during which the vehicle moves at a speed whose value is within a given interval is greater than or equal to a given time.

This step is similar to step 101 above and therefore will not be described further in detail.

Step 202: Entering said graphic and said radar data into the CNN model so that said model obtains information about the parking space, including its category, based on this data.

According to this embodiment of the present disclosure, a convolutional neural network model is installed in the electronic control unit, the training of which is carried out repeatedly and by training graphic and radar data collected from a large number of parking spaces of various types and entered into this model, and information about parking spaces of various types includes one or more of any of the following types of information: information about the presence of traffic cones in a parking space, information about the presence of signs prohibiting parking in a parking space, information about the presence of a parking barrier in a parking space, information about the presence of a parking space in a parking space markings and information about the presence of obstacles in the parking space. Further, this example implementation of the present disclosure does not place any restrictions on the choice of parking spaces, the range of which can be changed or expanded.

At this step, it is possible to enter graphic and radar data obtained by the electronic control unit into the given model of the convolutional neural network.

Further, the convolutional neural network model, having received graphic and radar data, using its hidden layers, extracts features of graphic and radar recognition from them, then classifies the extracted features and, accordingly, information about parking spaces using the output layers.

Among other things, information about a parking space may also contain information about its location and category. As a rule, parking spaces fall into the following categories: parallel, perpendicular and corner parking. On FIG. 6 is a schematic representation of the three-space parking classifier mentioned in the exemplary embodiment of the present disclosure, including, as seen in FIG. 6, "a" indicates the perpendicular parking area, "b" the parallel parking area, and "c" and "d" the corner parking areas. A parallel parking space is a parking space whose large edge is parallel to the carriageway. A perpendicular parking space is a parking space whose large edge is perpendicular to the carriageway. A corner parking space is a parking space whose larger edge is neither parallel nor perpendicular to the carriageway.

Step 203: outputting said convolutional neural network model to said parking space information, including its category information.

At this step, it is possible to output and send by the convolutional neural network model to the electronic control unit information about the parking space, including information about its category, after its classification.

Step 204: With reference to said corner parking space, said CNN model determines a parking direction based on said graphic and radar data if said parking space is a corner one.

Once a parking space has been determined by the CNN model as a corner parking space, as described in step 202, you can proceed directly to step 204.

At this step, it is possible for the convolutional neural network model to subsequently analyze the graphical and radar data associated with this parking space, after it is determined by the model as a corner parking space, and determine the direction in which the car should drive into this corner parking space.

Specifically, generally, a corner parking where the front of the car goes inward is called forward corner parking, and a corner parking where the front of the car is on the drive-in side of the parking space is called reverse corner parking.

Further, a distinction is made between corner packages that are driven forward and corner packages that are driven in the opposite direction. As seen in FIG. 6, in zone "c" there are places that are entered in the forward direction, and in zone "d" - places that are entered in the opposite direction. The place where the car is parked in the forward direction is characterized in that the angle enclosed between the larger edge of such a parking space and the direction of traffic in the carriageway is less than 90 degrees, and the place where the car is parked in the opposite direction is characterized in that the angle enclosed between the larger edge of such a parking space and the direction of movement in the carriageway exceeds 90 degrees.

Forward entry into a forward entry corner parking space requires only one small turn, while parking in a forward entry but reverse entry method requires multiple high angle turns. It follows from this that the exact determination of the direction of arrival at the corner parking space allows you to reduce the number of maneuvers in the subsequent planning of the route of movement during automatic parking and, as a result, reduce the likelihood of errors.

Further, if it follows from the information about the parking space category output by the CNN model that this parking space is a corner one, then the CNN model can subject the graphic and radar data corresponding to this parking space to subsequent analysis. After the direction of arrival to the corner parking space is determined, the convolutional neural network model evaluates this parking space according to three criteria.

Specifically, the evaluation criterion of the first degree is the priority determination by the convolutional neural network model of whether there is a sign indicating the direction of arrival in a given corner parking space, and if there is such a sign, then the car enters the parking space in the direction indicated by this sign. The evaluation criterion of the second degree is, if in this corner parking space there is no sign indicating the direction of arrival, the determination by the convolutional neural network model of whether other cars are parked on both sides of this parking space, and if there are other cars on both sides, and the direction of their arrival is the same, then the arrival of this car into the parking place is carried out in the same direction in which those other cars are parked. The criterion for evaluating the third degree is, if there are no other cars on either side of the given parking space or there are other cars, but they are parked in different directions, the determination by the convolutional neural network model of whether this corner parking space is a place that is entered in the forward direction, and if so, then the forward driving method is selected as the parking method, otherwise the reverse driving method is selected.

For example, a car pulls up to a parking lot to park, and the parking space is determined by a CNN model based on graphical and radar data as a corner parking space. In this case, the convolutional neural network model classifies the given parking space according to the parking direction. If a given corner parking space does not have a sign indicating the direction of arrival, but there are other cars on both sides, and these cars are parked in the same direction, then the assessment is carried out according to the criterion of the second degree. Further, if the arrival of both cars parked on both sides of the considered parking space is made in front, then the method of arrival at this corner parking space is determined by the convolutional neural network model as the method of arrival in front.

According to this example of the implementation of the present disclosure, it is possible, if the parking space is a corner one, to determine, using a convolutional neural network model, the method of entering this parking space, which would increase the practicality of the automated parking system for this method of maneuvering the car, reduce the number of maneuvers in the subsequent planning of the route during automatic parking, reduce the likelihood of errors and increase user comfort while driving.

In addition, information about the parking space output by said convolutional neural network model includes, among other things, information about the method of driving into said parking space if it is a corner one.

At this step, it is possible to output and send the convolutional neural network model to the electronic control unit information about the parking space, including information about the direction of arrival at this place, after this information has been classified.

Step 205: displaying a parking management interface containing information about at least one of said parking spaces, if at the time of the signal to search for a parking space, the current gear in which said vehicle is driving is said forward or said idle gear.

At this step, the user intending to automatically park the car can open the control interface of the car's automatic parking, thereby signaling the need to park. After receiving such a signal, the ECU receives information about the gear in which the vehicle is currently moving through the TCU, and if the current vehicle gear is forward or idle, the automatic parking control interface is replaced by the parking control interface. If parking space information is already stored in the ECU before the display unit displays the parking management interface, this information is displayed in the parking management interface. If there is no parking space information in the ECU before the display unit displays the parking management interface, the parking management interface reminds the user to keep driving the car and continuously collect graphic and radar data around the car with sensors until the parking space information is stored in the ECU and the information is displayed in the parking management interface.

Specifically, the parking management interface can display information about a plurality of parking spaces, one of which can be selected by the user according to his actual needs.

Step 206: Selecting information about the final parking space from the general flow of information about at least one parking space when a parking space selection command corresponding to the parking management interface is received.

At this step, it is possible for the user to select, in accordance with his actual needs, information about one parking space and the possibility for the electronic control unit to select information about the final parking space from the general flow of information about at least one parking space when a command is received to select a parking space corresponding to the parking management interface.

Specifically, in FIG. 7 is a flow diagram of the automated parking process referred to in an embodiment of the present disclosure. As seen in FIG. 7, after the vehicle is started, the parking system enters the parking space search standby mode 01, and when the vehicle speed meets the set criteria to start the parking space search, the vehicle starts to collect graphic and radar data and then proceeds to the parking space detection stage 02, but if the vehicle speed does not meet the set criteria to start the parking space search, the automated parking system again switches to the parking space search standby mode 01. the car is in forward gear or idle, the car monitor displays information about parking spaces 03, and when the user commands the action selection “Park” and the car moves again, the automated parking system re-executes the detection of parking spaces. ki 06 ends.

The optimal solution proposed by the present disclosure is that, in order to avoid parking the car in the wrong place due to erroneous actions of the user, when choosing information about the final parking place, it is required to apply the foot brake at the same time. Only in this way will the user's command to select the "Park" action be sent to the car, and the accuracy of the choice of information about the final parking space can be ensured.

Step 207: Generating a car parking path that said car is automatically parked based on said final parking location information.

This step is similar to step 104 above and therefore will not be described in detail further.

Step 208: command to temporarily stop the automatic parking process in case an abnormal state of said vehicle is detected.

At this step, it is possible to determine and send information about the working state of the car by sensors to the electronic control unit, which, based on the information received, determines whether this state of the car is abnormal, and if it is, gives the car a command to temporarily stop the automatic parking process. The state of the car is considered abnormal when the driver is not wearing a seat belt, the car doors are not closed and/or there is an obstacle on the parking path of the mentioned car.

Specifically, the seat belt sensor that the vehicle is equipped with can detect whether the driver is wearing a seat belt. If during automatic parking of the car, the seat belt sensor detects that the driver's seat belt is unfastened, then it sends this information to the electronic control unit, which, having received such information from the seat belt sensor, issues a command to temporarily stop the process of automatic car parking.

Specifically, the body control module with which the vehicle is equipped can determine whether the vehicle's doors are closed. If during automatic parking of the vehicle, the body control module detects that the vehicle doors are not closed, then it sends this information to the electronic control unit, which, having received such information from the body control module, issues a command to temporarily stop the process of automatic parking of the vehicle.

Specifically, the radar sensors with which the vehicle is equipped can determine whether there are obstacles in the parking path of said vehicle. If during the automatic parking of the car, the radar sensors detect that there are obstacles on the trajectory for parking the said car, they send this information to the electronic control unit, which, having received such information from the radar sensors, issues a command to temporarily stop the process of automatic parking of the car.

Further, the mentioned abnormal conditions may include other types of abnormal conditions in which the car may find itself, for example, if the driver's seat is additionally equipped with a sensor for the presence of a person in the driver's seat, then such a sensor can, in accordance with the result of measuring the pressure on the seat, send information to the electronic control unit about the absence of a person in the driver's seat when the user leaves this seat. This example implementation of the present disclosure does not impose any restrictions in this respect.

The optimal solution proposed by the present disclosure is that if the duration of the vehicle in the abnormal state exceeds the predetermined interruption time of the parking process, then the electronic control unit can independently exit the automated parking system in order to give the user enough time to troubleshoot and start the car again after its abnormal condition has been eliminated.

For example, if during the automatic parking of the car in the parking lot, one of the passengers sitting in the car opened the door himself, then the body control module, detecting the state of the car when its doors are not closed, sends this information to the electronic control unit, which confirms that the car is in an abnormal state, then issues a command to temporarily stop the automatic parking process and displays information to the user that the car is currently in an abnormal state. On FIG. 8 is a schematic representation of the vehicle in abnormal condition referenced in the exemplary implementation of this disclosure, including the inscription “Warning! The doors are not closed!” appeared in the parking management interface.

According to this embodiment of the present disclosure, the electronic control unit can determine with the help of sensors whether the vehicle is in an abnormal condition. If the current state of the vehicle is abnormal, the ECU may command to temporarily stop the automatic parking process of the vehicle. The ability, by detecting the working state of the car in real time when it is being automatically parked, detecting abnormal conditions of the car and suspending the automatic parking in a timely manner, can ensure the functional safety of the automated parking system and improve the user's comfort while driving the car.

Step 209: Commanding to continue the automatic parking process of said vehicle in case of confirmation that said abnormal state of said vehicle has been removed.

At this step, it is possible to determine by the sensors that the vehicle has exited the abnormal state after such a state of the vehicle has been eliminated by the user, while the sensors send information about this to the electronic control unit, which, based on the information received, finally determines whether the abnormal state of the vehicle has been eliminated, and if such a state has actually been eliminated, gives a command to continue the process of automatic parking of the vehicle.

Specifically, after the abnormal state of the car is cleared, an interface is displayed asking the user whether to continue to park the car automatically. On FIG. 9 is a schematic representation of the options mentioned in the exemplary implementation of the present disclosure, displayed after the parking process is suspended. As seen in FIG. 9, after the abnormal condition is eliminated, the car makes a reminder on the monitor in the form of the words "Continue to park?" and offers two answer options - "Yes" and "No", one of which the user can choose according to their actual needs.

On FIG. 10 is a diagram of the automatic parking suspension process mentioned in the exemplary embodiment of the present disclosure. As seen in FIG. 10, if the vehicle is in an abnormal state during automatic parking, the electronic control unit issues a command to temporarily stop the parking process. In the event that the time that the vehicle is in the abnormal state exceeds the set parking process interruption time, the electronic control unit may terminate the automatic parking process. After the abnormal state of the vehicle is cleared, the user can choose to either continue auto parking by pressing the "Continue Parking" button, or terminate automatic parking by pressing the "End Parking" button.

Based on the foregoing, the method of controlling an automated parking system presented by the present disclosure includes: graphic and radar data collected by sensors around a car moving at a speed that is within a predetermined interval, and the time during which the car is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; information about at least one parking space obtained from the output of a given model of a convolutional neural network, into which graphic and radar data are entered, and the training of the convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types; final parking space information selected from the at least one parking space information stream according to the parking space selection command received, if the current gear the vehicle is in is forward or idle; a trajectory for automatic parking of the car, generated based on the information about the final parking place. The present disclosure provides for the possibility of early search near the car of a parking space by measuring the speed of the car, identifying a parking space for parking a car in heterogeneous situations by applying a convolutional neural network model, in addition, it is possible to provide information about a plurality of free parking spaces, which allows satisfying the user's needs in searching for parking spaces of various types, optimizing the automated parking system itself and improving user comfort while driving.

In addition, the present disclosure provides for the ability to determine the method of entering a parking space if this parking space is a corner one, which would increase the practicality of the automated parking system for this method of maneuvering the car, reduce the number of maneuvers in the subsequent planning of the route of movement during automatic parking, reduce the likelihood of errors and increase user comfort while driving.

On FIG. 11 is a flow diagram of another method of controlling the automated parking system mentioned in the exemplary implementation of the present disclosure.

Step 301: determining the operating states of the image sensors, radar sensors and the electronic control unit when starting the car, and distinguish between good and emergency states;

This step provides for the launch of the automated parking system, which occurs simultaneously with the launch of the car. After starting the automated parking system, the electronic control unit begins to check the working condition of the image sensors, radar sensors and itself, and distinguishes between good and emergency states.

Specifically, in practice, electronic systems in automobiles are bulky and complex systems, especially sensors mounted on the outside of the bodies. Such sensors break easily and thus negatively affect the entire functionality of the car. In this regard, this embodiment of the present disclosure provides for the possibility for the electronic control unit to check the operating status of the image sensors, radar sensors, and itself after the start of the automated parking system.

The optimal solution proposed by the present disclosure is that the electronic control unit can also check the operating status of other sensors and electronic systems, which is not limited by this embodiment of the present disclosure.

Step 302: determining the functional category of said vehicle based on said operating states.

At this step, it is possible for the electronic control unit to determine the functional category of the automated car parking system in accordance with the operating states of various sensors.

More specifically, this step can be broken down into the following three substeps:

Step 3021: if the working state of all said image sensors, radar sensors and electronic control unit is healthy, then said vehicle functional category is determined as said normal operation category;

In this step, the functional category of the vehicle is defined as the category of normal operation if the working condition of all image sensors, radar sensors and the electronic control unit is good. In addition, during normal operation, the automated parking system ensures the normal operation of all image sensors, radar sensors and the electronic control unit.

Step 3022: if the operating state of at least one of said radar sensors and/or the electronic control unit is emergency, then said vehicle functional category is determined to be said emergency category of the first degree.

At this step, the functional category of the vehicle is determined as an emergency category of the first degree, if the operating state of at least one of the said radar sensors and/or the electronic control unit is emergency.

Step 3023: if the operating state of all said image sensors is emergency, then said vehicle functional category is determined to be said second degree emergency category.

At this step, the functional category of the car is determined as an emergency category of the second degree, if the working state of all the mentioned image sensors is emergency.

Step 3024: If the operating state of some of said image sensors is an emergency, then said vehicle functional category is determined to be said emergency category of the third degree.

At this step, the functional category of the vehicle is determined as an emergency category of the third degree, if the working state of some of the mentioned image sensors is emergency.

Upon completion of step 3022, step 303 is performed.

Step 303: issuing a command to terminate the operation of said electronic control unit if the functional category of said vehicle is determined to be said emergency category of the first degree.

At this step, it is possible to prohibit the functioning of the corresponding components of the automated parking system by the electronic control unit if the functional category of the car is defined as an emergency category of the first degree, so that the user cannot use this system.

Upon completion of step 3023, step 304 is performed.

Step 304: issuing a command to stop the operation of said image sensors if the functional category of said vehicle is determined to be said emergency category of the second degree.

At this step, it is possible to prohibit the automatic parking system from turning on all image sensors if the functional category of the car is defined as an emergency category of the second degree, and allow the system to turn on only radar sensors and an electronic control unit.

Specifically, due to the prohibition of turning on image sensors, the use of a convolutional neural network model is not possible. While the vehicle is processing the parking space information, the ECU can use the data from the radar sensors to create a virtual map centered on the vehicle and continuously updated with new radar data until the parking space information is received.

Upon completion of step 3024, step 305 is performed.

Step 305: issuing a command to stop the operation of those image sensors whose said operating state is determined to be said emergency state, if the functional category of said vehicle is determined to be said third degree emergency category.

At this step, it is possible for the automated parking system to disable the activation of those image sensors whose working state is defined as emergency, if the functional category of the car is defined as an emergency category of the third degree, and to allow the system to turn on only those image sensors whose working state is defined as good, while the model of the convolutional neural network of the electronic control unit can be used, however, its accuracy will be reduced to some extent.

Step 306: using said sensors to collect graphical and radar data around said vehicle moving at a speed that is within a predetermined interval, and the time during which the vehicle is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time.

This step is similar to step 101 above and therefore will not be described in detail further.

Step 307: inputting said graphical and radar data into a given CNN model and obtaining information about at least one parking space as an output of said model, wherein said CNN model is trained on the basis of training graphical and radar data collected from a large number of parking spaces of various types.

This step is similar to step 102 above and therefore will not be described further in detail.

Step 308: Selecting, from the general flow of information about at least one parking space, information about the final parking space according to the received parking space selection command, if the current gear in which the vehicle is traveling is forward or idle.

This step is similar to step 103 above and therefore will not be described in detail further.

Step 309: Generate a car parking path that said car automatically parks based on said final parking location information.

This step is similar to step 104 above and therefore will not be described further in detail.

Based on the foregoing, the method of controlling an automated parking system presented by the present disclosure includes: graphic and radar data collected by sensors around a car moving at a speed that is within a predetermined interval, and the time during which the car is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; information about at least one parking space obtained from the output of a given model of a convolutional neural network, into which graphic and radar data are entered, and the training of the convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types; final parking space information selected from the at least one parking space information stream according to the parking space selection command received, if the current gear the vehicle is in is forward or idle; a trajectory for automatic parking of the car, generated based on the information about the final parking place. The present disclosure provides for the possibility of early search near the car of a parking space by measuring the speed of the car, identifying a parking space for parking a car in heterogeneous situations by applying a convolutional neural network model, in addition, it is possible to provide information about a plurality of free parking spaces, which allows satisfying the user's needs in searching for parking spaces of various types, optimizing the automated parking system itself and improving user comfort while driving.

In addition to the above, the present disclosure also provides for the ability to adjust the levels of the emergency category, to determine the working state of the automated parking system and the degree of the emergency category using the electronic control unit, to activate certain functions in the automated parking system in accordance with a certain degree of emergency category, and to improve the practicality of the automated parking system and the user's driving comfort by helping the user to park the car even when some of the functions have ceased to work.

On FIG. 12 is a flow diagram of another method of controlling the automated parking system mentioned in the exemplary implementation of the present disclosure.

Step 401: Collecting by means of sensors graphic and radar data around said vehicle moving at a speed whose value is within a given interval, and the time during which the vehicle moves at a speed whose value is within said limits of a given interval is greater than or equal to a given time.

This step is similar to step 101 above and therefore will not be described in detail further.

Step 402: Inputting said graphical and said radar data into a given CNN model and outputting said CNN model information about at least one parking space, said CNN model being trained based on training graphics and radar data collected from a large number of different types of parking spaces.

This step is similar to step 102 above and therefore will not be described further in detail.

Step 403: selecting from the general flow of information about at least one parking space information about the final parking space in accordance with the received parking space selection command, if the current gear in which the car is moving is forward or idle.

This step is similar to step 103 above and therefore will not be described in detail further.

Step 404: generation of a car parking path that said car automatically parks based on said final parking location information.

This step is similar to step 104 above and therefore will not be described further in detail.

Step 405: Collecting said environmental data around said vehicle with said sensors if said vehicle's current gear is in parking gear at the time of the signal to search for a parking space.

At this step, the user who intends to automatically park the car can open the automatic parking control interface, thereby signaling the need to park. Having received such a signal, the electronic control unit through the TCU receives information about the gear in which the car is currently moving, and if the current car gear is park, then the radar sensors collect radar data about the environment around the car.

More specifically, since the trajectory of a car leaving a parking space is more simplified than the trajectory of entering a parking space, the presence of radar data is sufficient to determine the direction of the car's exit.

Step 406: Confirmation of at least one driving direction of said vehicle to parking spaces based on environmental data.

At this step, it is possible for the ECU to obtain specific information about the distance from the vehicle to obstacles, if any, from the general environment data, and confirm the direction of the vehicle's exit from the parking space in accordance with the received environmental data.

More specifically, environmental data can be either radar or graphical. If the distance from the car to the obstacles present around it is greater than the specified value, the car starts the automatic parking program. After turning on the automatic parking function, the electronic control unit creates a virtual map of the area based on the available radar data, according to which it determines the direction the car will leave the parking place. The direction in which the vehicle exits the parking space can be one or a combination of several of the following directions: forward exit, reverse exit, left exit, and right exit.

For example, if at the moment when the user intends to leave the parking lot, there is a road to the left of the place where his car is parked, there is a wall to the right of the specified place, there are other cars behind him, and there are no cars in front, then the electronic control unit creates a virtual map of the area based on the available radar data, which determines the following combination of directions for the exit of this car from the parking lot: exit forward with a turn to the left.

Step 407: Displaying the vehicle exit control interface in which at least one of said parking exit directions is indicated.

At this step, after the ECU selects the direction of the vehicle exit to the parking spaces, it switches from the automatic exit control interface to the vehicle exit control interface from the parking space, in which the exit direction is indicated.

More specifically, in FIG. 13 is a diagram of a direction selection for exiting a parking space provided by the automated parking system mentioned in the exemplary implementation of the present disclosure, including a car exit management interface displaying four exit directions identified by the car around it, from which the user can select according to their actual needs.

Step 408: Selecting the final direction from said exit directions when commanded to leave the parking space.

At this step, it is possible for the electronic control unit to select from the available directions the final direction for driving the car out of the parking place when a command is received from the user to leave the parking lot.

Step 409: Generating a car exit path that said car will automatically exit the parking lot based on said information about the final exit direction of the parking lot.

At this step, it is possible for the electronic control unit to generate a trajectory for the car to leave the parking place based on information about the final direction of the exit, while the electronic control unit, interacting with the ESP, EPS and TCU in controlling the direction of movement, as well as the processes of gear shifting, braking and stopping, automatically takes the car out of the final parking space along the generated trajectory.

Specifically, the electronic control unit can compare the actual location of the vehicle with the end point corresponding to the final exit direction from the parking space, calculate the relative distance between the vehicle and the end point, and draw up the necessary coordinate system based on the calculation result. Such a coordinate system will allow the electronic control unit to determine the location from the coordinates, the turning radius and the angle enclosed between the car and the final parking space in order to calculate the trajectory of the exit from the parking lot. Before the car starts to automatically park, the ESP can continuously monitor the speed and braking processes of the vehicle by ECU commands, the EPS can continuously monitor the vehicle's direction by ECU commands, and the TCU can continuously monitor gearshifts by ECU commands until the vehicle is fully driven into its final parking space and the automatic parking process is completed.

The optimal solution proposed by the present disclosure is that the ECU can similarly command to temporarily stop the automatic parking process if the process detects that the vehicle is in an abnormal condition, and resume the process after such an abnormal condition is removed.

On FIG. 14 is a diagram of another type of automated parking process mentioned in the exemplary implementation of the present disclosure. As seen in FIG. 14, at the time when the car is waiting for a command from the user to select the direction of exit from the parking lot, the electronic control unit can confirm the transition to the stage of preparation for leaving the parking place by the command to select the action "Leave the parking lot", while the electronic control unit interacts with all other systems and, if the interaction goes smoothly, the car starts leaving the parking space, and if such interaction fails, then the process of automatically leaving the parking lot stops. If the interaction of the electronic control unit and all other systems goes smoothly, then the car starts to automatically leave the parking place. If the car is in an abnormal state, the electronic control unit issues a command to temporarily stop the exit from the parking lot. In the event that the time that the vehicle is in the abnormal state exceeds the predetermined exit process interruption time, the ECU may end the automatic parking exit process. After the abnormal state of the car is cleared, the user can choose to either continue automatic parking exit by pressing the Continue Automatic Parking Exit button, or terminate automatic parking by pressing the End Automatic Parking Exit button.

Based on the foregoing, the method of controlling an automated parking system presented by the present disclosure includes: graphic and radar data collected by sensors around a car moving at a speed that is within a predetermined interval, and the time during which the car is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; information about at least one parking space obtained from the output of a given model of a convolutional neural network, into which graphic and radar data are entered, and the training of the convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types; final parking space information selected from the at least one parking space information stream according to the parking space selection command received, if the current gear the vehicle is in is forward or idle; a trajectory for automatic parking of the car, generated based on the information about the final parking place. The present disclosure provides for the possibility of early search near the car of a parking space by measuring the speed of the car, identifying a parking space for parking a car in heterogeneous situations by applying a convolutional neural network model, in addition, it is possible to provide information about a plurality of free parking spaces, which allows satisfying the user's needs in searching for parking spaces of various types, optimizing the automated parking system itself and improving user comfort while driving.

In addition to the above, the present disclosure also provides for the ability of the automated parking system to plan the trajectory of the vehicle when leaving the parking space based on the available radar data, thereby relieving the user of the need to drive the car to exit the parking lot, optimizing the automated parking system itself and increasing user comfort while driving.

Based on the above implementation example, the present disclosure also provides devices for implementing a method for controlling an automated parking system.

On FIG. 15 shows a design drawing of the device mentioned in the implementation example of the present disclosure for implementing the method of controlling the automated parking system, optionally containing the following modules:

The first data acquisition module 501 used to collect, using said sensors, graphic and radar data around said vehicle moving at a speed that is within a predetermined interval, and the time during which the vehicle is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time;

A neural network module 502 used to input said graphical and said radar data into a given CNN model and to output said CNN model information about at least one parking space, wherein said CNN model is trained based on training graphics and radar data collected from a large number of different types of parking spaces;

A "Park" action selection module 503, used to select information about the final parking location from the general flow of information about at least one said parking space in accordance with the incoming command to select a parking space, if the current gear in which said vehicle is driving is forward or idle;

A path selection module 504 used to generate a parking path for a car on which said car is automatically parked based on said parking destination information.

Optionally, the device presented by the present disclosure for implementing the method of controlling the automated parking system includes, among other things:

A measurement module 505 used to determine the operating states of said image sensors, radar sensors and an electronic control unit when starting said vehicle, distinguishing between good and fault states;

A functional category determination module 506 used to determine the functional category of said vehicle based on said operating states.

Optionally, said functional category definition module 506 includes:

The first confirmation sub-module 5061, used to confirm that the functional category of said vehicle is determined to be said normal operation category, if the working state of said image sensors, radar sensors and electronic control unit is normal;

The second confirmation sub-module 5062, used to confirm that the functional category of said vehicle is determined to be said emergency category of the first degree, if the operating state of at least one of said radar sensors and/or electronic control unit is emergency;

A third confirmation sub-module 5063, used to confirm that the functional category of said vehicle is determined to be said second degree emergency category if the operating state of all said image sensors is emergency;

The fourth confirmation sub-module 5064, used to confirm that the functional category of said vehicle is determined to be said emergency category of the third degree, if the operating state of some of said image sensors is emergency.

Optionally, the device presented by the present disclosure for implementing the method of controlling the automated parking system includes, among other things:

The first control module 507, whose role is to issue a command to stop the operation of said electronic control unit in the event that the functional category of said vehicle is determined to be said emergency category of the first degree;

The second control module 508, whose role is to issue a command to stop the operation of said image sensors in the event that the functional category of said vehicle is determined to be said emergency category of the second degree;

The third control module 509, whose role is to issue a command to stop the operation of those image sensors whose operating state is determined to be said emergency state, in case the functional category of said vehicle is determined to be said third degree emergency category.

Optionally, the information about said parking space includes one or more of any of the following types of information: information about the presence of traffic cones in the parking space, information about the presence of signs prohibiting parking in the parking space, information about the presence of a parking barrier in the parking space, information about the presence of markings in the parking space, and information about the presence of obstacles in the parking space.

Optionally, said neural network module 502 includes:

An input sub-module 5021 used to input said graphic and said radar data into the CNN model and obtain by said CNN model information about said parking space, including its category;

The first output submodule 5022 used to output said convolutional neural network model information about said parking space, including its category;

Optionally, said neural network module 502 includes:

Corner parking spot parking direction determination sub-module 5023, used to determine by said CNN model with respect to said corner parking spot a parking direction based on said graphic and radar data if said parking spot is a corner one; said parking space information output from said convolutional neural network model includes, but is not limited to, parking direction information for said corner parking space.

Optionally, said "Park" action selection module 503 includes:

A display sub-module 5031 used to display a parking management interface containing information about at least one of said parking spaces if, at the time of the signal to search for a parking space, the current gear in which said vehicle is running is said forward or said idle gear;

Selection sub-module 5032 used to select information about the final parking space from the general flow of information about at least one said parking space upon receipt of a command to select a parking space corresponding to said parking control interface.

Optionally, devices presented by this disclosure for implementing the method of controlling an automated parking system include, among other things:

The second data acquisition module 510, used to collect said environment data around said vehicle using said sensors, if said vehicle's current gear is parked at the time of the signal to search for a parking space;

A confirmation module 511 used to confirm at least one driving direction of said vehicle to parking spaces based on said environmental data;

A display module 512 used to display a vehicle exit control interface comprising at least one of said exit directions;

An "Exit Parking Lot" action selection module 513 used to select, from said parking exit directions, a final exit direction according to the parking exit command received;

A parking exit trajectory determination module 514 used to generate a trajectory for a vehicle leaving a parking space, along which said vehicle, based on said final exit direction information, automatically exits the parking space.

Optionally, the device presented by the present disclosure for implementing the method of controlling the automated parking system includes, among other things:

A parking abort module 515 used to issue a command to temporarily abort the automatic parking process in case an abnormal state of said vehicle is detected;

A parking resuming module 516 used to command the continuation of the automatic parking process of said vehicle in the event that said abnormal state of said vehicle is confirmed to be corrected.

Optionally, it is possible to determine that the vehicle is in the aforementioned abnormal state if at least one of the following circumstances occurs:

Said vehicle is in the state where the driver is not wearing a seat belt;

Said vehicle is in a state where its doors are not closed;

An obstacle appears on the trajectory for parking the mentioned car.

Based on the foregoing, the device for implementing the method of controlling the automated parking system, represented by the present disclosure, includes: graphic and radar data collected by sensors around a car moving at a speed that is within a predetermined interval, and the time during which the car is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; information about at least one parking space obtained from the output of a given model of a convolutional neural network, into which graphic and radar data are entered, and the training of the convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types; final parking space information selected from the at least one parking space information stream according to the parking space selection command received, if the current gear the vehicle is in is forward or idle; a trajectory for automatic parking of the car, generated based on the information about the final parking place. The present disclosure provides for the possibility of early search near the car of a parking space by measuring the speed of the car, identifying a parking space for parking a car in heterogeneous situations by applying a convolutional neural network model, in addition, it is possible to provide information about a plurality of free parking spaces, which allows satisfying the user's needs in searching for parking spaces of various types, optimizing the automated parking system itself and improving user comfort while driving.

Those skilled in the art will readily appreciate that, for convenience and ease of description, the specific principles of operation of the above systems, devices, and assemblies are similar to those given in the exemplary implementation of the method of the present disclosure, and will not be re-described hereinafter.

The above example is an example of the best use of the present disclosure, but does not set any boundaries in its implementation. Any changes, adequate replacements or improvements made to this disclosure without violating its main idea and principles are subject to patent rights and are subject to protection within the scope of this application.

The above example is only a specific example of the implementation of the present disclosure and does not set any boundaries in its implementation. Any modifications or substitutions that may be conceived by technicians knowledgeable in this technology field, and which do not go beyond the technical scope of this disclosure, are subject to patent rights and are subject to protection within the scope of this application, and therefore, as the boundaries of patent rights in relation to this disclosure, the boundaries of patent rights in relation to the Claims given in this application should be taken.

The above example of the implementation of devices serves only as a means of illustration, and under the separation of elements described as separate parts, both physical and non-physical separation can be meant, elements that serve to display blocks can be both physical and non-physical blocks, i.e. elements can be located in one place, or they can be distributed over different network blocks. The tasks posed by this example implementation of the disclosure may be performed using all or part of the modules selected in accordance with the actual needs of the implementer. Technical workers in this field can understand and implement this example without creative work.

Examples of implementation of various parts of the present disclosure may be implemented through hardware or software modules operating in one or more processors, or both simultaneously. Technicians in the art should understand that all or part of the functions of all or part of the elements of the computing and processing equipment presented in the exemplary implementation of the present disclosure may in practice be performed by microprocessors or digital signal processors (DSPs). In addition, programs (eg, computer programs and their products) of all or part of the equipment or devices used to implement the method described herein may also be implemented within the scope of this disclosure. Such programs for implementing the present disclosure may be stored in a machine-readable medium or be in the form of one or more signals that can be downloaded from Internet sites, provided on signal carriers, or in any other form.

For example, in FIG. 16 shows a computer processing equipment with which the method of the present disclosure is implemented. Conventionally, such computing equipment consists of a processor 1010 and a computer program or machine readable medium in the form of a storage device 1020. The storage device 1020 can be an electronic storage device such as a flash memory, EEPROM (Electrically Erasable Reprogrammable Read Only Memory), EPROM (Semiconductor Programming Memory), hard disk, or ROM (Read Only Memory). Memory 1020 has space 1030 for storing program codes 1031 used to implement any of the steps of applying the method described above. For example, program code storage space 1030 may contain program codes 1031 used to implement each step of applying the proposed method. These program codes may be read or entered from one or more computer program products into one or more computer program products, which include program code media such as hard disks, compact discs (CDs), memory cards, or magnetic disks. The most commonly used as such a computer program product is the portable or fixed storage unit illustrated in FIG. 17. Such a storage unit may have the same storage segment or space similar to storage 1020 in the computing processing equipment shown in FIG. 16. Program codes can be compressed accordingly. Typically, the memory blocks contain machine readable codes 1031', i. e. codes read by processors like processor 1010. By processing these codes, the computing/processing equipment implements all the steps of applying the method described above.

The expressions "an implementation example" or "one or more implementation examples" appearing in the text mean that the specific features, structures, or characteristics collected in the description of the example/examples are combined into at least one implementation of the present disclosure. In addition, you should pay attention to the fact that the expression "in an implementation example" occurring in the text does not mean that there is only one such example and that it is the same example in each case.

This application provides many specific details, however, it is acceptable if the applications of the objects of the present disclosure will be implemented without taking into account such details. In order not to complicate the process of understanding the contents of the present application, in the exemplary implementation of the disclosure presented therein, well-known methods, structures and technologies are not given in detail.

None of the referential symbols enclosed in brackets in the claims should be used as criteria for establishing restrictions in relation to it. The expression "consist of" or "comprise" does not exclude the possibility of the existence of components or operations other than those listed in the Claims. The naming of one or another component in the singular does not mean at all that other similar components no longer exist. The present disclosure may be implemented by hardware with various components and a computer with a suitable programming language. The various installations listed in the claims may be represented by the same apparatus. The expressions "first", "second", "third", etc. do not point to any sequence. Such expressions can be understood as denominations.

Finally, it should be noted that this example implementation serves only to describe the technical solutions presented in this disclosure, and does not place any restrictions on them. Despite the fact that the above implementation example provides a detailed description of the present disclosure, ordinary technical workers in this field should understand that they can make certain changes to the technical solutions presented in the above implementation example, or make an adequate replacement of part of the technical features, if such changes or replacements do not disappear the essence and do not go beyond the limits of those technical solutions that the implementation example of the disclosure in this patent application offers.

Claims (65)

1. Method of control by an automated parking system, characterized in that it is applied to cars equipped with sensors, and includes: graphical and radar data collected by said sensors around a vehicle moving at a speed that is within a predetermined interval, and the time during which the vehicle is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; information about at least one parking space obtained at the output of a given convolutional neural network model, into which said graphic and radar data are entered, and the training of said convolutional neural network model is carried out on the basis of training graphic and radar data collected from a large number of parking spaces of various types; if the current gear in which the vehicle is traveling is forward or idle, then the final parking space information is selected from the total flow of at least one parking space information in accordance with the parking space selection command received; on the basis of said information about the final parking place, a trajectory is generated for parking the car, along which the said car is automatically parked, and said sensors mean a plurality of image sensors and a plurality of radar sensors, and said vehicle is equipped with an electronic control unit, and before graphic and radar data are collected by said sensors around said vehicle under conditions when said vehicle moves at a speed whose value is within a predetermined interval, and the time during which the vehicle moves at a speed whose value is within a predetermined interval is greater than or equal to a predetermined time, this method is also characterized as follows: when starting said vehicle, the operating state of said image sensors, radar sensors, and electronic control unit is determined, wherein good and emergency states are distinguished; on the basis of said operating states, the functional category of said vehicle is determined, wherein the functional category of said vehicle can be any of four (normal operation category, first degree emergency category, second degree emergency category, and third degree emergency category), and said step of determining the functional category of said vehicle based on said operating states is characterized as follows: if the operating condition of said image sensors, radar sensors and electronic control unit is good, then said functional category of the vehicle is defined as said category of normal operation; if the operating state of at least one of said radar sensors and/or electronic control unit is emergency, then said functional category of the vehicle is defined as said emergency category of the first degree; if the operating state of all said image sensors is in an emergency, then said functional category of the vehicle is determined as said emergency category of the second degree; if the operating state of some of said image sensors is in emergency, then said functional category of the vehicle is defined as said emergency category of the third degree. 2. The method according to claim 1, characterized in that said electronic control unit is used to control the process of automatic parking of said vehicle, and after the step of determining the functional category of said vehicle based on said operating states is carried out: if the functional category of said vehicle is determined to be said emergency category of the first degree, then the system issues a command to terminate the operation of said electronic control unit; if the functional category of said vehicle is determined to be said emergency category of the second degree, then the system issues a command to terminate the operation of said image sensors; if the functional category of said vehicle is determined to be said emergency category of the third degree, then the system issues a command to stop the operation of those image sensors whose said operating state is determined to be said emergency state. 3. The method according to claim 1, characterized in that the information about said parking space includes one or more of the following types of information: information about the presence of traffic cones in the parking space, information about the presence of signs prohibiting parking in the parking space, information about the presence of a parking barrier in the parking space, information about the presence of markings in the parking space and information about the presence of obstacles in the parking space. 4. The method according to claim 1, characterized in that the step at which the mentioned graphic and radar data are entered into a given convolutional neural network model to obtain information about at least one parking space at the output of said convolutional neural network model includes the following steps: determining, by said convolutional neural network model, said information about the parking space, including its category, based on said graphical and radar data entered into said convolutional neural network model; obtaining at the output of said convolutional neural network model said information about the parking space, including information about the category of the parking space. 5. The method according to claim 4, characterized in that, after the stage is completed, at which the mentioned model of the convolutional neural network, based on the mentioned graphic and radar data entered into it, determines information about the parking space, including its category: if said parking space is a corner one, then said convolutional neural network model determines for said corner parking space a parking direction based on said graphical and radar data; said parking space information output from said convolutional neural network model includes, but is not limited to, parking direction information for said corner parking space. 6. The method according to claim. 1, characterized in that the step at which information about the final parking place is selected from the general flow of information about at least one said parking space in accordance with the received command to select a parking space, if the current gear in which the said vehicle is moving is forward or idle, includes the following steps: if, at the time the parking space search signal is received, the current gear in which said vehicle is running is said forward gear or said idle gear, a parking control interface is displayed containing information about at least one said parking space; upon receipt of a command to select a parking space corresponding to said parking control interface, information about the final parking space is selected from the general information flow about at least one said parking space in accordance with the received command to select a parking space. 7. The method according to claim 1, characterized in that, after the step is completed, at which, based on the information about the final parking place, a trajectory is generated for parking the car, along which the said car is automatically parked: if at the moment of receiving the signal to search for a parking space, the current transmission of said vehicle is parking, using said sensors, said environmental data around said vehicle is collected; based on said environmental data, at least one exit direction of said vehicle is determined from the parking space; displaying a vehicle exit control interface in which at least one of said parking exit directions is indicated; upon receipt of a command to leave the parking place, on its basis, the final direction is selected from the mentioned exit directions; on the basis of said information about the final direction of the exit from the parking lot, a trajectory for leaving the car from the parking lot is generated, along which the said car will automatically leave the parking place. 8. The method according to claim 1, characterized in that after the step is completed, at which, based on the information about the final parking place, a trajectory is generated for parking the car, along which the said car is automatically parked: if an abnormal state of said vehicle is detected, the system issues a command to temporarily stop the automatic parking process; if it is confirmed that said abnormal state of said vehicle has been eliminated, the system instructs to continue the automatic parking process. 9. The method according to claim 8, characterized in that the car is considered to be in the said abnormal state if at least one of the following circumstances occurs: said vehicle is in a state where the driver is not wearing a seat belt; said vehicle is in a state where its doors are not closed; an obstacle appears on the trajectory for parking the mentioned car. 10. A device for implementing a method for controlling an automated parking system, characterized in that it includes: a first data acquisition module used to collect, using said sensors, graphic and radar data around said vehicle moving at a speed that is within a predetermined interval, and the time during which the vehicle is moving at a speed that is within a predetermined interval is greater than or equal to a predetermined time; a neural network module used to input said graphical and said radar data into a given CNN model and to output said CNN model information about at least one parking space, wherein said CNN model is trained based on training graphics and radar data collected from a large number of different types of parking spaces; a "Park" action selection module used to select information about the final parking location from the general flow of information about at least one said parking space in accordance with the incoming command to select a parking space, if the current gear in which said vehicle is running is forward or idle; a path selection module used to generate a path for parking the vehicle along which said vehicle is automatically parked based on said parking destination information; a measurement module used to determine the operating states of said image sensors, radar sensors and an electronic control unit when starting said vehicle, distinguishing between good and fault states; a functional category determination module used to determine the functional category of said vehicle based on said operating states, moreover, the mentioned module for determining the functional category includes: a first confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said normal operation category if the operating state of said image sensors, radar sensors and electronic control unit is normal; a second confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said emergency category of the first degree, if the operating state of at least one of said radar sensors and/or the electronic control unit is emergency; a third confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said second degree emergency category if the operating state of all said image sensors is emergency; a fourth confirmation sub-module used to confirm that the functional category of said vehicle is determined to be said emergency category of the third degree, if the operating state of some of said image sensors is emergency. 11. The device according to claim 10, characterized in that it includes: a first control module whose role is to issue a command to stop the operation of said electronic control unit in case the functional category of said vehicle is determined to be said emergency category of the first degree; a second control module, the role of which is to issue a command to stop the operation of said image sensors in the event that the functional category of said vehicle is determined to be said emergency category of the second degree; the third control module, the role of which is to issue a command to stop the operation of those image sensors, the said operating state of which is determined as the mentioned emergency condition, in case the functional category of the said vehicle is determined as the mentioned emergency category of the third degree. 12. Devices according to claim 10, characterized in that they include, among other things: a second data acquisition module used to collect said environmental data around said vehicle with said sensors if said vehicle's current gear is parked at the moment a parking space search signal is received; a confirmation module used to confirm at least one driving direction of said vehicle from the parking space based on the environment data; a display unit used to display a vehicle exit control interface including at least one of said exit directions; an "Exit Parking Space" action selection module used to select from said parking space exit directions a final exit direction in accordance with the parking exit command; a parking exit trajectory determination module used to generate a trajectory for the vehicle to leave the parking space, along which said vehicle, based on said information about the final exit direction, will automatically leave the parking space. 13. Computing and processing equipment, characterized in that it includes: a storage device that stores machine-readable codes; one or more processors, during the processing of machine-readable codes, by which or by which the computing processing equipment implements the method of controlling the automated parking system referred to in any of paragraphs. 1-9. 14. A machine-readable environment in which a computer program is stored that contains machine-readable codes and provides for the implementation by the computing and processing equipment of the method of controlling the automated parking system mentioned in any of paragraphs. 1-9 when the machine readable codes are processed by said computer processing equipment.