KR20240044573A - Apparatus for controlling autonomous vehicle and method thereof - Google Patents

Abstract

The present invention relates to a control device for an autonomous vehicle, comprising: a sensor module including at least one sensor for detecting the operation and behavior state of the autonomous vehicle; a camera module including at least one camera installed toward the front or side of the autonomous vehicle; a GPS module that receives signals from GPS satellites and detects the current location of the autonomous vehicle; and a processor that controls steering of the autonomous vehicle and performs a lane keeping operation based on information detected through the sensor module, the camera module, and the GPS module.

Description

Control device and method for autonomous vehicle {APPARATUS FOR CONTROLLING AUTONOMOUS VEHICLE AND METHOD THEREOF}

The present invention relates to a control device and method for an autonomous vehicle. More specifically, in the event of a situation where the lane on the road on which an autonomous vehicle is traveling and the driving field of view are unsecured, an alternative image is obtained and used from a driving image database, and the vehicle occupants and It relates to a control device and method for an autonomous vehicle that ensures the safety of pedestrians.

In general, self-driving vehicles (or self-driving cars) use precise maps, satellite navigation systems (GPS), and other vehicle sensors (e.g. lidar, radar, cameras, etc.) even if the driver does not operate the steering wheel, accelerator pedal, or brakes. ) refers to a car that understands the surrounding situation and navigates to its destination on its own.

The stages of autonomous driving technology are defined by the Society of Automotive Engineers (SAE) as international standards and are classified into a total of 6 stages from stage 1 to stage 5.

Among these, Adaptive Cruise Control (ACC) and Lane Keeping Assist System (LKAS) are representative technologies in Stage 1, and the number of vehicles equipped with Advanced Driver Assistance Systems (ADAS) has recently been increasing.

At this time, the autonomous driving may or may not be performed smoothly depending on road conditions or weather (snow, rain, etc.).

In other words, in the past, when the field of view of the camera image is unsecured due to whiteout, night, snow, rain, etc., it is impossible for autonomous vehicles to maintain a normal distance between vehicles and maintain the center of the lane, which can lead to accidents that threaten human safety. There is a problem.

Accordingly, there is a need for technology that can ensure the safety of vehicle occupants and pedestrians by securing and using alternative images from the driving image database in situations where lanes and driving visibility are unsecured.

The background technology of the present invention is disclosed in Republic of Korea Patent Publication No. 10-2021-0100776 (published on August 18, 2021, vehicle performing autonomous driving using a camera and method of operating the same).

According to one aspect of the present invention, the present invention was created to solve the above problems, and in the event of a situation where the lane on the road on which an autonomous vehicle is traveling and the driving field of view are not secured, an alternative image is secured and used from a driving image database. The purpose is to provide a control device and method for an autonomous vehicle that ensures the safety of vehicle occupants and pedestrians.

A control device for an autonomous vehicle according to an aspect of the present invention includes a sensor module including at least one sensor for detecting the operation and behavior state of the autonomous vehicle; a camera module including at least one camera installed toward the front or side of the autonomous vehicle; a GPS module that receives signals from GPS satellites and detects the current location of the autonomous vehicle; and a processor that controls steering of the autonomous vehicle and performs a lane keeping operation based on information detected through the sensor module, the camera module, and the GPS module.

In the present invention, the processor monitors the front through the sensor module to detect road curvature and a preceding vehicle, analyzes images captured through the camera module, performs a lane maintenance operation, and detects a specified number of satellite signals or more. When received, the road on which the vehicle is currently traveling and the direction in which it will be driven are estimated from the navigation map.

In the present invention, the processor communicates with a database of a designated server, and transmits and stores real-time driving image data captured by the camera module to the database, or obtains and receives driving image data stored in the database, The driving image data stored in the database is characterized in that vehicle location and lane information are matched.

In the present invention, the driving image data stored in the database is characterized in that it includes driving image data stored in other vehicles or autonomous vehicles.

In the present invention, the server can delete the driving image data whose lane and visibility are below the standards among the driving image data stored in the database, and add a score to the driving image data whose lane and visibility are above the standards. , It is characterized in that it is implemented so that only a specified percentage of the driving image data or a specified number of the driving image data for the same route can be stored.

In the present invention, the processor calculates the first steering value of the vehicle based on lane information detected through the sensor module, processes the image captured through the camera module, and calculates the first steering value of the vehicle based on the lane information detected through the sensor module. A second steering value is calculated, a third steering value corresponding to the direction in which the vehicle is to proceed is calculated through a navigation map based on the GPS signal detected through the GPS module, and the first to third steering values are calculated. Compare with each other, exclude the steering value with the largest difference, calculate the average steering value of the remaining two steering values excluding the excluded steering value, and control the autonomous vehicle using the average steering value of the two steering values. It is characterized by:

In the present invention, the processor acquires real-time driving image data from the camera module and transmits and stores it to a designated database, and determines the current location of the autonomous vehicle from among the previously stored image data from the designated database. Obtain image data corresponding to the real-time driving image data in real time, analyze the real-time driving image data, and if it is possible to secure the lane and driving view, perform autonomous driving control by processing the real-time driving image data, and determine the lane from the real-time driving image data. And when it is impossible to secure a driving field of view, autonomous driving control is performed by acquiring and processing driving image data corresponding to real-time driving information of the current autonomous vehicle from among the image data stored in the database.

A method for controlling an autonomous vehicle according to another aspect of the present invention includes: detecting, by a processor, the operation and behavior state of the autonomous vehicle through a sensor module; capturing, by the processor, an image of the front or side of the autonomous vehicle through a camera module; detecting, by the processor, the current location of the autonomous vehicle through a GPS module; and controlling, by the processor, the steering of the autonomous vehicle based on information detected through the sensor module, the camera module, and the GPS module to perform a lane keeping operation. do.

In the present invention, the processor monitors the front through the sensor module to detect road curvature and a preceding vehicle, analyzes images captured through the camera module, performs a lane maintenance operation, and detects a specified number of satellite signals or more. When received, the road on which the vehicle is currently traveling and the direction in which it will be driven are estimated from the navigation map.

In the present invention, after the step of capturing an image of the front or side of the autonomous vehicle, the processor communicates with a database of a designated server and transmits real-time driving image data captured by the camera module to the database for storage. Alternatively, the driving image data stored in the database can be obtained and received, and the driving image data stored in the database is characterized in that the vehicle's location and lane information are matched.

In the present invention, the driving image data stored in the database is characterized in that it includes driving image data stored in other vehicles or autonomous vehicles.

In the present invention, the server can delete the driving image data whose lane and visibility are below the standards among the driving image data stored in the database, and add a score to the driving image data whose lane and visibility are above the standards. , It is characterized in that it is implemented so that only a specified percentage of the driving image data or a specified number of the driving image data for the same route can be stored.

In the present invention, in order to control the steering of the autonomous vehicle, the processor calculates the first steering value of the vehicle based on lane information detected through the sensor module and images captured through the camera module. Processes to calculate a second steering value of the vehicle based on the detected lane information, and calculates a third steering value corresponding to the direction in which the vehicle will proceed through a navigation map based on the GPS signal detected through the GPS module, Compare the first to third steering values, exclude the steering value with the largest difference, calculate the average steering value of the remaining two steering values excluding the excluded steering value, and calculate the average steering value of the two steering values. It is characterized by controlling an autonomous vehicle using average steering values.

In the present invention, in order to control the steering of the autonomous vehicle, the processor acquires real-time driving image data from the camera module and transmits and stores it to a designated database, and from the designated database, pre-stored data is obtained. Among the video data, video data corresponding to the current location of the self-driving vehicle is acquired in real time, and the real-time driving video data is analyzed to control autonomous driving by processing the real-time driving video data when it is possible to secure the lane and driving field of view. If it is impossible to secure the lane and driving field of vision from the real-time driving image data, obtain and process driving image data corresponding to the real-time driving information of the current autonomous vehicle from among the image data stored in the database to perform autonomous driving control. It is characterized by:

According to one aspect of the present invention, in the event of a situation where the lane on the road on which an autonomous vehicle is traveling and the driving field of view are unsecured, the present invention secures and uses alternative images from a driving image database to ensure the safety of vehicle occupants and pedestrians. do.

1 is an exemplary diagram showing the schematic configuration of a control device for an autonomous vehicle according to an embodiment of the present invention.
Figure 2 is a flowchart illustrating a control method for an autonomous vehicle according to a first embodiment of the present invention.
Figure 3 is a flowchart illustrating a control method for an autonomous vehicle according to a second embodiment of the present invention.

Hereinafter, an embodiment of a control device and method for an autonomous vehicle according to the present invention will be described with reference to the attached drawings.

In this process, the thickness of lines or sizes of components shown in the drawing may be exaggerated for clarity and convenience of explanation. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, definitions of these terms should be made based on the content throughout this specification.

1 is an exemplary diagram showing the schematic configuration of a control device for an autonomous vehicle according to an embodiment of the present invention.

As shown in FIG. 1, the control device for an autonomous vehicle according to this embodiment includes a sensor module 110, a camera module 120, a processor 130, a steering control module 140, and a GPS module 150. ) includes.

The sensor module 110 includes at least one sensor for detecting the operation and behavior state of the vehicle.

For example, the sensor module 110 includes a wheel speed sensor, a yaw rate sensor, a steering angle sensor, an acceleration sensor, an ultrasonic sensor, a radar sensor, a lidar sensor, and an infrared sensor. It may include etc.

The camera module 120 includes at least one camera installed toward the front or side of the vehicle.

The GPS module 150 can receive signals from GPS satellites and detect the current location of the vehicle.

The processor 130 performs a lane keeping operation based on information detected through the sensor module 110, the camera module 120, and the GPS module 150.

In addition, the processor 130 monitors the front through the sensor module 110, detects road curvature and preceding vehicles, and analyzes images captured through the camera module 120 to perform lane maintenance operation. In addition, when more than a specified number (e.g., 3) of satellites are detected (i.e., when more than a specified number of satellite signals are received), the processor 130 determines the road on which the vehicle is currently traveling from the navigation map (electronic map). You can estimate (predict) the driving direction.

For example, if the current driving road is a straight road (or a straight section), the processor 130 can predict that the driving direction is a straight road (or a straight section), and if the road ahead is a curved road, the driving direction is a curved direction. (or rotation section) can be estimated (predicted).

In addition, the processor 130 communicates with an external (e.g. designated server) database (e.g., driving image database) (not shown), and stores real-time driving image data captured by the camera module 120 in the database (e.g., It can be delivered to and stored in the database (e.g., driving image database) (not shown), or it can be delivered by acquiring (loading) the driving image data stored in the database (e.g., driving image database) (not shown).

At this time, the driving image data stored in the database (e.g., driving image database) (not shown) includes location and lane information, and accordingly, driving image data corresponding to the current location of the vehicle can be acquired (loaded). .

In addition, the driving image data stored in the database (e.g., driving image database) (not shown) includes driving image data stored by other vehicles (or autonomous vehicles), and the server (not shown) is stored in the database (e.g., Among the driving image data stored in the driving image database (not shown), driving image data whose lane and visibility are below the standards can be deleted, and points are added to the driving image data whose lane and visibility are above the standards (e.g. (A higher score is given to driving video data with high lane and visibility), and space in the database can be secured by storing only the driving video data of the higher specified ratio (or the higher specified number) of the driving video data for the same route. You can.

In addition, the processor 130 calculates the steering value (first steering value) of the vehicle based on the lane information detected through the sensor module 110, and processes the image captured through the camera module 120. The steering value (second steering value) of the vehicle is calculated based on the detected lane information, and the steering value (second steering value) corresponding to the direction in which the vehicle will proceed is calculated through the navigation map based on the GPS signal detected through the GPS module 150. 3 steering values), and the optimal steering value can be calculated based on the three steering values (first to third steering values).

For example, the processor 130 compares the three steering values (first to third steering values), excludes the value with the largest difference, and calculates the average steering value of the remaining two steering values to drive autonomous driving. You can control the vehicle.

The steering control module 140 controls steering according to the control of the processor 130, and controls not to turn in a specific direction during a designated time, or between the start point of the designated time and the end point of the designated time. It can be controlled so that the steering occurs gradually.

-First Embodiment-

Figure 2 is a flowchart for explaining a control method of an autonomous vehicle according to the first embodiment of the present invention.

Referring to FIG. 2, the processor 130 acquires real-time driving image data from the camera module 120 (S101) and transmits the real-time driving image data to a designated database (e.g., driving image database) (not shown). Do it (S102).

At this time, the image data transmitted to the database (e.g., driving image database) (not shown) is matched with real-time driving information (e.g., location, lane information, etc.) of the current autonomous vehicle and transmitted.

In addition, the processor 130 obtains (loads) image data corresponding to the current location of the autonomous vehicle in real time from the designated database (e.g., driving image database) (not shown) among pre-stored image data. (S103).

In addition, the processor 130 analyzes the real-time driving image data to check whether the lane and driving field of view can be secured (S104), and if the lane and driving field of view can be secured from the real-time driving image data (example in S104), Real-time driving image data obtained from the camera module 120 is processed to perform autonomous driving control (S105).

Meanwhile, if it is impossible to secure the lane and driving field of view from the real-time driving image data (No in S104), the processor 130 selects the current autonomous vehicle vehicle from among the image data stored in the database (e.g., driving image database) (not shown). Autonomous driving control is performed by acquiring (loading) and processing image data corresponding to real-time driving information (e.g., location, lane information, etc.) (S106).

At this time, the processor 130 controls the steering control module 140 to perform the autonomous driving control.

-Second Embodiment-

Figure 3 is a flowchart for explaining a control method of an autonomous vehicle according to a second embodiment of the present invention.

Referring to FIG. 3, the processor 130 calculates the steering value (first steering value) of the vehicle based on the lane information detected through the sensor module 110 (S201), and the camera module 120 The image captured through is processed to calculate the vehicle's steering value (second steering value) based on the detected lane information (S202), and the navigation map based on the GPS signal detected through the GPS module 150 is calculated. A steering value (third steering value) corresponding to the direction in which the vehicle will travel is calculated (S203).

And the processor 130 can control the autonomous vehicle by calculating the optimal steering value based on the three steering values (first steering value to third steering value).

For example, the processor 130 compares the three steering values (first to third steering values), excludes the steering value with the largest difference (S204), and excludes the two remaining steering values excluding the excluded steering values. The average steering value is calculated (S205), and the autonomous vehicle is controlled using the average steering value of the two steering values (S206).

As described above, in this embodiment, normal autonomous driving is possible even in situations where lanes and driving visibility are unsecured due to whiteout, night, snow, rain, etc., thereby ensuring human safety, and the self-camera camera is not a problem of the surrounding environment. Even when a system error occurs, normal autonomous driving is possible, which has the effect of maintaining autonomous driving performance and ensuring human safety.

As described above, the present invention has been described with reference to the embodiments shown in the drawings, but these are merely illustrative, and various modifications and equivalent embodiments can be made by those skilled in the art. You will understand the point. Therefore, the scope of technical protection of the present invention should be determined by the scope of the patent claims below. Implementations described herein may also be implemented as, for example, a method or process, device, software program, data stream, or signal. Although discussed only in the context of a single form of implementation (eg, only as a method), implementations of the features discussed may also be implemented in other forms (eg, devices or programs). The device may be implemented with appropriate hardware, software, firmware, etc. The method may be implemented in a device such as a processor, which generally refers to a processing device that includes a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices such as computers, cell phones, portable/personal digital assistants (“PDAs”) and other devices that facilitate communication of information between end-users.

110: sensor module
120: Camera module
130: processor
140: Steering control module
150: GPS module

Claims (14)

A sensor module including at least one sensor for detecting the operation and behavior state of the autonomous vehicle;
a camera module including at least one camera installed toward the front or side of the autonomous vehicle;
a GPS module that receives signals from GPS satellites and detects the current location of the autonomous vehicle; and
A processor that controls steering of the autonomous vehicle and performs a lane keeping operation based on information detected through the sensor module, the camera module, and the GPS module. controller.
The method of claim 1, wherein the processor:
Monitors the front through the sensor module to detect road curvature and vehicles ahead, performs lane maintenance by analyzing images captured through the camera module, and detects the vehicle from the navigation map when more than a specified number of satellite signals are received. A control device for an autonomous vehicle, characterized in that it estimates the road on which it is currently traveling and the direction in which it will travel.
The method of claim 1, wherein the processor:
By communicating with the database on the designated server,
Real-time driving image data captured by the camera module can be transmitted and stored in the database, or the driving image data stored in the database can be obtained and delivered,
The driving image data stored in the database is,
A control device for an autonomous vehicle, characterized in that vehicle location and lane information are matched.
According to clause 3,
The driving image data stored in the database is,
A control device for an autonomous vehicle, characterized in that it includes driving image data stored from another vehicle or an autonomous vehicle.
The method of claim 3, wherein the server:
Among the driving image data stored in the database, driving image data that does not meet the standards for lane and visibility can be deleted,
Autonomous driving, characterized in that it is implemented by adding points to driving image data with lane and visibility exceeding the standard, and storing only a specified percentage or a specified number of driving image data among the driving image data for the same route. The vehicle's control unit.
The method of claim 1, wherein the processor:
Calculate a first steering value of the vehicle based on lane information detected through the sensor module, and calculate a second steering value of the vehicle based on lane information detected by processing images captured through the camera module, Calculate a third steering value corresponding to the direction in which the vehicle will proceed through a navigation map based on the GPS signal detected through the GPS module,
Compare the first to third steering values, exclude the steering value with the largest difference, calculate the average steering value of the remaining two steering values excluding the excluded steering value, and calculate the average steering value of the two steering values. A control device for an autonomous vehicle, characterized in that it controls an autonomous vehicle using an average steering value.
The method of claim 1, wherein the processor:
Obtain real-time driving video data from the camera module and transmit and store it in a designated database,
From the designated database, acquire image data corresponding to the current location of the autonomous vehicle in real time among pre-stored image data,
By analyzing the real-time driving image data, if it is possible to secure the lane and driving field of view, perform autonomous driving control by processing the real-time driving image data,
When it is impossible to secure the lane and driving field of view from the real-time driving image data, autonomous driving control is performed by acquiring and processing driving image data corresponding to real-time driving information of the current autonomous vehicle from among the image data stored in the database. A control device for an autonomous vehicle.
detecting, by a processor, an operation and behavioral state of the autonomous vehicle through a sensor module;
capturing, by the processor, an image of the front or side of the autonomous vehicle through a camera module;
detecting, by the processor, the current location of the autonomous vehicle through a GPS module; and
Characterized in that it comprises a step of, by the processor, controlling steering of the self-driving vehicle based on information detected through the sensor module, the camera module, and the GPS module to perform a lane keeping operation. Control method for autonomous vehicles.
The method of claim 8, wherein the processor:
Monitors the front through the sensor module to detect road curvature and vehicles ahead, performs lane maintenance by analyzing images captured through the camera module, and detects the vehicle from the navigation map when more than a specified number of satellite signals are received. A control method for an autonomous vehicle, characterized by estimating the road on which it is currently traveling and the direction in which it is to be driven.
According to clause 8,
After taking images of the front or side of the autonomous vehicle,
The processor,
By communicating with the database on the designated server,
Real-time driving image data captured by the camera module can be transmitted and stored in the database, or the driving image data stored in the database can be obtained and delivered,
The driving image data stored in the database is,
A control method for an autonomous vehicle, characterized in that vehicle location and lane information are matched.
According to clause 10,
The driving image data stored in the database is,
A control method for an autonomous vehicle, characterized in that it includes driving image data stored from another vehicle or an autonomous vehicle.
The method of claim 10, wherein the server:
Among the driving image data stored in the database, driving image data that does not meet the standards for lane and visibility can be deleted,
Autonomous driving, characterized in that it is implemented by adding points to driving image data with lane and visibility exceeding the standard, and storing only a specified percentage or a specified number of driving image data among the driving image data for the same route. Vehicle control methods.
The method of claim 8, wherein to control steering of the autonomous vehicle,
The processor,
Calculate a first steering value of the vehicle based on lane information detected through the sensor module, and calculate a second steering value of the vehicle based on lane information detected by processing images captured through the camera module, Calculate a third steering value corresponding to the direction in which the vehicle will proceed through a navigation map based on the GPS signal detected through the GPS module,
Compare the first to third steering values, exclude the steering value with the largest difference, calculate the average steering value of the remaining two steering values excluding the excluded steering value, and calculate the average steering value of the two steering values. A control method for an autonomous vehicle, characterized in that the autonomous vehicle is controlled using an average steering value.
The method of claim 8, wherein to control steering of the autonomous vehicle,
The processor,
Obtain real-time driving video data from the camera module and transmit and store it in a designated database,
From the designated database, acquire image data corresponding to the current location of the autonomous vehicle in real time among pre-stored image data,
By analyzing the real-time driving image data, if it is possible to secure the lane and driving field of view, perform autonomous driving control by processing the real-time driving image data,
When it is impossible to secure the lane and driving field of view from the real-time driving image data, autonomous driving control is performed by acquiring and processing driving image data corresponding to real-time driving information of the current autonomous vehicle from among the image data stored in the database. A control method for an autonomous vehicle.

Images