KR102656811B1 - System and method for controlling autonomous driving vehicle based on multiple sensor data - Google Patents

Abstract

The present disclosure provides an autonomous driving control method based on a plurality of sensor data, executed by at least one processor. This method includes receiving a plurality of first sensing data from a plurality of sensors, converting the domain of the plurality of first sensing data to generate a plurality of second sensing data, and comparing results of the plurality of second sensing data. Based on this, it may include detecting a sensor that generates abnormal sensing data among a plurality of sensors, and outputting a warning message when a sensor that generates abnormal sensing data is detected.

Description

Autonomous driving control system and method based on multiple sensor data {SYSTEM AND METHOD FOR CONTROLLING AUTONOMOUS DRIVING VEHICLE BASED ON MULTIPLE SENSOR DATA}

The present disclosure relates to an autonomous driving control system and method based on a plurality of sensor data, and more specifically, by comparing input data regarding surrounding information detected by heterogeneous sensors to detect sensors with problems with input data, thereby ensuring safety. It relates to an autonomous driving system and method that can perform efficient autonomous driving.

An autonomous driving system may use multiple heterogeneous sensors, multiple power supplies, or multiple computing components to ensure reliable driving and safety. For example, when multiple computing components (or processors) process the same sensor data to generate autonomous driving control signals, detect the processor in which an operation error occurred by comparing the autonomous driving control signals, and respond appropriately. By executing the procedure, more reliable autonomous driving can be achieved. In addition, various software and hardware enhancements are being made to operate autonomous vehicles effectively and stably.

Because malfunctions in autonomous driving systems can cause casualties and significant property damage, manufacturers are required to take responsibility for their design. To this end, autonomous driving systems are being developed to respond appropriately to fatal errors or failures even in limited situations, rather than being implemented to respond well to all situations on average.

For example, in order to realize the reliability of autonomous driving systems, technologies are being developed to strengthen hardware elements so that system components can withstand extreme situations (high heat, high pressure, overcurrent, shock, etc.). In addition, methods for monitoring the operating status of each component in real time, such as conducting self-defect tests to detect errors in circuits or software within the system, are being studied.

In general, in detecting sensor defects in an autonomous driving system, there are many methods for detecting defects in the sensor's internal circuit or hardware and software related to the sensor, such as BIST (Built-in Self-Test). However, even if the sensor is in normal operating condition, there are adverse road environments (e.g. inside a tunnel, bad weather, night situations, direct sunlight toward the driver, etc.) or obstacles that interfere with sensor detection (e.g. attached to the sensor area). If the sensor input is incorrect due to foreign substances, etc.), the sensor may be incorrectly recognized as malfunctioning, or incorrect information about the surrounding situation may be input to the autonomous driving system.

Meanwhile, among conventional autonomous driving technologies, various methods have been proposed to analyze and classify data acquired by sensors using artificial neural networks as a method to increase the accuracy and reliability of recognition of the surrounding environment in various situations. However, if there are actual obstacles outside the sensor, it may be determined that the sensor is defective even though there is no defect in the sensor itself, and data from the sensor determined to be defective can be excluded. In other cases, if the artificial neural network includes incorrectly input sensor data in the calculations instead of excluding it, autonomous driving control signals may be generated inappropriately, increasing the risk of accidents.

The present disclosure provides an autonomous driving control system and method based on a plurality of sensor data to solve the problems of the conventional autonomous driving system described above.

According to an embodiment of the present disclosure, an autonomous driving control method based on a plurality of sensor data, executed by at least one processor, includes receiving a plurality of first sensing data from a plurality of sensors, a plurality of first sensing Converting the domain of data to generate a plurality of second sensing data, detecting a sensor generating abnormal sensing data among the plurality of sensors based on a comparison result of the plurality of second sensing data, and detecting a sensor generating abnormal sensing data An autonomous driving control method may be included, including outputting a warning message when a sensor generating data is detected.

According to one embodiment, based on a comparison result of a plurality of second sensing data, the step of detecting a sensor that generates abnormal sensing data according to environmental factors of the autonomous vehicle among the plurality of sensors includes a plurality of second sensing data. An autonomous driving control method may be included, including detecting a sensor that generates sensing data with the lowest similarity to other sensing data.

According to one embodiment, based on a comparison result of a plurality of second sensing data, the step of detecting a sensor that generates abnormal sensing data according to environmental factors of the autonomous vehicle among the plurality of sensors includes a plurality of second sensing data. It may include detecting a sensor that generates sensing data whose difference from other sensing data exceeds a predetermined threshold.

According to one embodiment, when a sensor generating abnormal sensing data is detected, the step of outputting a warning message includes outputting a warning message requesting confirmation whether the abnormal sensing data is due to environmental factors of the autonomous vehicle. It can be included.

According to one embodiment, the autonomous driving control method may further include switching the driving mode of the autonomous vehicle to a semi-autonomous driving mode or a manual driving mode when a sensor generating abnormal sensing data is detected.

According to one embodiment, the autonomous driving control method may further include controlling the autonomous vehicle to drive on an emergency driving path when a sensor generating abnormal sensing data is detected.

According to one embodiment, the plurality of sensors may include at least one of a radar sensor, a LiDAR sensor, an infrared sensor, and a camera.

According to one embodiment, the step of generating a plurality of second sensing data by converting the domain of the plurality of first sensing data includes converting the plurality of first sensing data into image data of a two-dimensional domain to generate the plurality of second sensing data. It may include generating sensing data.

According to one embodiment, an autonomous driving control system based on a plurality of sensor data includes at least one processor connected to a memory and configured to execute at least one computer-readable program included in the memory, and at least one program Receives a plurality of first sensing data from a plurality of sensors, converts the domain of the plurality of first sensing data to generate a plurality of second sensing data, and based on the comparison result of the plurality of second sensing data , may include instructions for detecting a sensor generating abnormal sensing data among the plurality of sensors and outputting a warning message when a sensor generating abnormal sensing data is detected.

According to various embodiments of the present disclosure, the autonomous driving system can identify sensor input errors caused by changes in the external environment or obstacles and take appropriate responses to them. Through this, the efficiency and reliability of autonomous driving can be improved by reducing the possibility that the autonomous driving system may use power unnecessarily or cause an accident by operating with unnecessary or incorrect responses to sensor input errors.

According to various embodiments of the present disclosure, the safety and reliability of the autonomous driving system can be further improved by determining not only whether there is a sensor input error but also the current state of the driver and executing an appropriate response. In addition, by providing an autonomous driving method suitable for implementing safe and rapid movement, which is one of the purposes of the autonomous driving system, it can help passengers reach their destination comfortably and safely.

According to various embodiments of the present disclosure, in a method for a driver or passenger of a vehicle using an autonomous driving system to respond to a system defect, whether the operation error of the autonomous driving system is due to a serious internal defect or the driver can simply respond to it. By providing guidance to identify possible defects in the system, it can help drivers themselves take appropriate action against system defects.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, in which like reference numerals indicate like elements, but are not limited thereto.
1 is a diagram illustrating the configuration of an autonomous driving system according to an embodiment of the present disclosure and an example of sensor data input error due to external obstacles.
Figure 2 is a block diagram showing the configuration of an autonomous driving control system according to an embodiment of the present disclosure.
Figure 3 shows an example of a method for identifying abnormal sensing data by comparing data received from a plurality of heterogeneous sensors according to an embodiment of the present disclosure.
Figure 4 is a diagram illustrating an example of a method for determining the current status of a driver (passenger) according to an embodiment of the present disclosure.
Figure 5 is a flowchart of an autonomous driving control method based on a plurality of sensors according to an embodiment of the present disclosure.
Figure 6 is a flowchart of an autonomous driving control method based on a plurality of sensors according to another embodiment of the present disclosure.
Figure 7 is a flowchart of an autonomous driving control method based on a plurality of sensors according to another embodiment of the present disclosure.

Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, detailed descriptions of well-known functions or configurations will be omitted if there is a risk of unnecessarily obscuring the gist of the present disclosure.

In the accompanying drawings, identical or corresponding components are given the same reference numerals. Additionally, in the description of the following embodiments, overlapping descriptions of identical or corresponding components may be omitted. However, even if descriptions of components are omitted, it is not intended that such components are not included in any embodiment.

Advantages and features of the disclosed embodiments and methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the present disclosure is complete and to those skilled in the art to which the present disclosure pertains. It is provided only to fully inform the user of the scope of the invention.

Terms used in the present disclosure will be briefly described, and the disclosed embodiments will be described in detail.

The terminology used in this disclosure selects general terms that are currently widely used as much as possible while considering the functions in this disclosure, but this may vary depending on the intention or precedent of a technician working in the related field, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

In this disclosure, singular expressions include plural expressions, unless the context clearly specifies the singular. Additionally, plural expressions include singular expressions, unless the context clearly specifies plural expressions.

When it is said that a part 'includes' a certain element throughout the present disclosure, this means that other elements may be further included rather than excluding other elements, unless specifically stated to the contrary.

The term 'unit' or 'module' used in the specification refers to a software or hardware component, and the 'unit' or 'module' performs certain roles. However, 'part' or 'module' is not limited to software or hardware. A 'unit' or 'module' may be configured to reside on an addressable storage medium and may be configured to run on one or more processors. Thus, as an example, 'part' or 'module' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, etc. , procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. Components and 'parts' or 'modules' can be combined into smaller numbers of components and 'parts' or 'modules' or into additional components and 'parts' or 'modules'. Could be further separated.

According to an embodiment of the present disclosure, a 'unit' or 'module' may be implemented with a processor and memory. The term “processor” should be interpreted broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some contexts, “processor” may refer to an application-specific integrated circuit (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), etc. The term “processor” refers to a combination of processing devices, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such combination of configurations. It may also refer to

In this disclosure, 'system' may refer to a configuration that includes one or more servers, computing devices, software modules or components that execute the functions described in this disclosure, or a combination thereof, for example, It may refer to, but is not limited to, one or more computing devices, server devices, or distributed computing devices that provide cloud services. Hereinafter, when describing embodiments of an autonomous driving system that includes a function for detecting a sensor with an error in the input, the system includes a software architecture and operation that implements an autonomous driving system that detects a sensor with an error in the input. It may mean a system, library, driver, or a combination of these and one or more components of a hardware module such as a processor, memory, etc.

1 is a diagram illustrating the configuration of an autonomous driving system according to an embodiment of the present disclosure and an example of sensor data input error due to external obstacles.

As shown, the autonomous vehicle 110 may include an autonomous driving module 120 that generates an autonomous driving control signal by processing sensor data related to surrounding conditions of the vehicle. The autonomous driving module 120 can receive data (hereinafter referred to as ‘sensing data’) from a plurality of heterogeneous sensors including a camera sensor 130, radar 140, LiDAR 150, etc., An autonomous driving control signal can be generated based on the received sensing data. In Figure 1, three types of sensors such as cameras, radars, and lidar are shown as examples, but the type or number of sensors is not limited to this and may further include other types of sensors such as infrared sensors, sound wave sensors, etc. You can. In addition, the autonomous driving module 120 includes one or more processors capable of processing one or more sensing data, a memory capable of storing sensing data, and a data bus or input/output interface capable of connecting a plurality of sensors, processors, and memories. It may be a computing device that

FIG. 1 shows an example of an input error occurrence situation 100 in which the camera sensor 130 receives abnormal data due to an obstacle 160 (for example, a foreign substance such as a leaf). Normal sensing data 170 input by the camera sensor 130 can be used as suitable data for the autonomous driving module 120 to generate an autonomous driving control signal, but abnormal sensing data 180 is used to generate an autonomous driving control signal. cannot be used as appropriate data.

Abnormal sensing data 180 may occur due to an obstacle 160 not only to the camera 130 but also to other types of sensors such as radar 140 or lidar 150. The autonomous driving module 120 may detect abnormal sensing data 180 by comparing the sensing data received from each sensor. When detecting abnormal sensing data 180, the autonomous driving module 120 determines whether the error in the sensing data 180 is caused by an external obstacle 160 or a poor road environment (e.g., bad weather, direct sunlight, night situation, etc.). Alternatively, it can be determined whether it is due to a defect in the sensor itself.

In one embodiment, the autonomous driving module 120 executes a Built-In Self-Test (BIST) on a plurality of sensors 130, 140, and 150 to determine its own hardware among the plurality of sensors 130, 140, and 150. Alternatively, it can be determined whether there is a defect in the software. If a sensor defect according to BIST is not found, the autonomous driving module 120 detects an error in the sensing data 180 due to an external obstacle 160 or a poor road environment (e.g., bad weather, direct sunlight, night situation, etc.). It is determined that this is caused by, and the sensor generating abnormal sensing data 180 can be detected according to a method described in detail below. In another embodiment, the autonomous driving module does not execute a procedure to determine whether there is a defect in its own hardware or software among the plurality of sensors 130, 140, and 150, but generates abnormal sensing data ( 180) can be detected.

In one embodiment, the autonomous driving module 120 converts the domain of the sensing data (hereinafter referred to as 'first sensing data') received from a plurality of sensors 130, 140, and 150 to a plurality of second sensing data. Data can be generated. For example, the autonomous driving module 120 converts the sensing data received from the laser sensor 140 and the lidar sensor 150 into two-dimensional image data (or second sensing data), so that the camera sensor 130 It can have the same domain as the two-dimensional image data (or second sensing data) that is the sensing data received from. The autonomous driving module 120 may detect a sensor that generates abnormal sensing data among the plurality of sensors 130, 140, and 150, based on a comparison result of the plurality of second sensing data.

In the example shown in FIG. 1, since there are no external obstacles around the laser sensor 140 and the lidar sensor 150, the first sensing data received from the corresponding sensors is converted into the domain of the second sensing data. In this case, the second sensing data may have similar characteristics to the normal sensing data 170. On the other hand, since there is an external obstacle 160 around the camera sensor 160, the second sensing data obtained by domain converting the first sensing data received from the sensor has similar characteristics to the abnormal sensing data 180. You can. The autonomous driving module 120 converts the domain of a plurality of first sensing data received from a plurality of sensors 140, 150, and 160 to select a plurality of second sensing data with the lowest similarity with other sensing data. Sensing data may be judged to be abnormal. Accordingly, the autonomous driving module 120 may determine that an input error due to the obstacle 160 has occurred with respect to the sensor (eg, camera sensor 130) that generated the abnormal sensing data.

In one embodiment, when a sensor that generates abnormal sensing data is detected, the autonomous driving module 120 may output a warning message to be provided to the driver. Here, the warning message output by the autonomous driving module 120 may be a warning signal by sound, vibration, or light. Additionally, the warning message may be a warning message requesting confirmation whether abnormal sensing data is due to environmental factors of the autonomous vehicle. Additionally, when a sensor generating abnormal sensing data is detected, the autonomous driving module 120 generates an autonomous driving control signal that switches the driving mode of the autonomous vehicle 110 to a semi-autonomous driving mode or a manual driving mode. You can. Alternatively, the autonomous driving module 120 may generate an autonomous driving control signal that causes the autonomous vehicle 110 to travel on an emergency driving path when a sensor generating abnormal sensing data is detected.

Additionally, the autonomous driving module 120 may determine the driver's current state and generate a warning message and/or autonomous driving control signal according to the identified driver's state. The autonomous driving module 120 can determine the driver's current state using a camera or an infrared sensor (not shown) installed inside the autonomous vehicle 110.

In one embodiment, if the driver's current state is sleeping, drowsy, or drunk, the autonomous driving module 120 may output a warning message and execute an emergency driving mode. In the emergency driving mode, the autonomous driving module 120 may generate an emergency driving path and generate an autonomous driving control signal to drive the vehicle 110 along the generated emergency driving path. For example, an emergency driving route can be created considering the current location of the vehicle and surrounding conditions (vehicle, presence of shoulder), etc.

In another embodiment, when the driver's current state is an alert state, the autonomous driving module 120 may output a warning message and execute a semi-autonomous driving mode or a manual driving mode. In the semi-autonomous driving mode, the autonomous driving module 120 may perform autonomous driving using some sensors that generate normal sensing data among the plurality of sensors 130, 140, and 150.

According to the above embodiments, safe and reliable autonomous driving is achieved in response to sensing data errors by effectively detecting the occurrence of a sudden external obstacle 160 that causes errors in the input of various sensors installed in the autonomous vehicle. It can be run.

Figure 2 is a block diagram showing the configuration of an autonomous driving control system according to an embodiment of the present disclosure.

The autonomous driving control system 200 shown in FIG. 2 may be, for example, a system included in the autonomous vehicle 110 or the autonomous driving module 120 shown in FIG. 1 . As shown, the autonomous driving control system 200 processes the sensor unit 210, which includes a plurality of heterogeneous sensors 130, 140, and 150, and the sensing data received from the sensor unit 210 to perform an autonomous driving algorithm. It may include a processing unit 220 that performs processing, a comparison unit 230 that compares received sensing data, and an abnormal response unit 240 that responds to detection of abnormal sensing data or a sensor that generates it. For example, the processing unit 220, the comparison unit 230, and the abnormal response unit 240 may be included in the autonomous driving module 120 shown in FIG. 1.

In one embodiment, the sensor unit 210 may include a plurality of heterogeneous sensors necessary for autonomous driving, that is, a camera 130, a radar 140, and a lidar 150. A plurality of sensing data generated by the sensor unit 210 may be transmitted to the processing unit 220 through the sensor bus (BUS) 260.

In one embodiment, the processing unit 220 may include a sensor bus interface 222 and a calculation unit 224. The sensor bus interface 222 may receive a plurality of sensing data transmitted from the sensor unit 210 through the BUS 260. The calculation unit 224 may include a travel control unit 226 and a surrounding monitoring unit 228. The driving control unit 226 may generate an autonomous driving control signal based on the received sensing data. Additionally, the surrounding monitoring unit 228 may generate surrounding information data about the situation around the vehicle based on the received sensing data.

In one embodiment, the comparison unit 230 may include a domain conversion unit 232, an image matching unit 234, and a memory 236. The domain conversion unit 232 converts the domain of the sensing data received from the processing unit 220 to match the domain of the sensing data. For example, the domain converter 232 may convert the domain of the received sensing data into a two-dimensional image domain. The image matching unit 234 can identify abnormal sensing data by comparing sensing data with matched domains.

In one embodiment, the image matching unit 234 may determine that a sensor that outputs sensing data showing a similarity difference greater than or equal to a certain threshold as a result of comparing sensing data with matched domains is abnormal.

In one embodiment, the memory 236 of the comparison unit 230 may store a domain conversion algorithm, an image matching algorithm, and similarity threshold data. The memory 236 is shown as being disposed outside the domain conversion unit 232 and the image matching unit 234, but is not limited thereto, and depending on the implementation, the memory 236 may be disposed outside the domain conversion unit 232 and the image matching unit 234. It can be implemented inside each.

In one embodiment, the abnormality response unit 240 may include a warning unit 242, a driving mode determination unit 244, and an emergency travel path creation unit 246. The abnormality response unit 240 may execute a response procedure based on whether there is abnormal sensing data received from the comparison unit 230 or sensor information that generates abnormal sensing data.

In one embodiment, the warning unit 242 of the abnormal response unit 240 generates a warning message to be provided to the driver (or passenger) when receiving sensor information generating abnormal sensing data from the comparison unit 230. can do. For example, a warning message may be a warning signal by sound, vibration, or light. When receiving sensor information that generates abnormal sensing data from the comparison unit 230, the driving mode determination unit 244 generates a control signal to change the driving mode of the autonomous vehicle to semi-autonomous driving mode or manual driving mode. can do.

In one embodiment, the driving mode determination unit 244 identifies the current state of the driver and sends a control signal to switch the driving mode of the autonomous vehicle to a semi-autonomous driving mode or a manual driving mode according to the identified driver's state. can be created. When the driving mode determination unit 244 determines the driver's state, for example, it can determine whether the driver is drowsy by identifying blinking in the eye area among the driver's image captured by a camera sensor installed inside the vehicle. there is. If the driving mode determination unit 244 determines that the driver's current state is drowsy, a control signal for switching the current driving mode to the emergency driving mode may be generated and transmitted to the processing unit 220.

In one embodiment, when the driving mode determination unit 244 determines that the driver's current state is an alarm state, the driving mode determination unit 244 outputs a warning message through the warning unit 242 and changes the driving mode to a semi-autonomous driving mode or a manual driving mode. A switching control signal can be generated and transmitted to the processing unit 220. In response, the driving control unit 226 of the processing unit 220 may execute a semi-autonomous driving mode using some of the plurality of sensors that operate normally and do not receive abnormal input signals.

In one embodiment, when the driving mode determination unit 244 determines that the driver's current state is sleeping, drowsy, or drunk, it outputs a warning message through the warning unit 242 and the emergency driving path creation unit 246. You can perform emergency driving mode based on the path created in . The emergency travel route generator 246 may generate an emergency travel route by considering the current location of the vehicle and surrounding conditions (vehicles, presence of a shoulder, etc.). The emergency driving path generated by the emergency driving path generator 246 may mean a path from the current location of the autonomous vehicle to a safe location such as a parking lot, a sleepy rest area, a rest area, etc.

Figure 3 shows an example of a method for identifying abnormal sensing data by comparing data received from a plurality of heterogeneous sensors according to an embodiment of the present disclosure.

The image data 310, 320, 330, and 340 shown in FIG. 3 represents the surrounding environment (e.g., pedestrian priority road) of the autonomous vehicle detected by a plurality of sensors, and is processed by the processing unit 220 into 2 Indicates the state converted into an image of the dimensional domain. The processing unit 220 receives sensing data through the sensor bus interface 222, and in order to compare the received sensing data, the domain conversion unit 232 converts the domains of each sensing data to match them. For example, the domain converter 232 may convert the domains of a plurality of sensing data into a two-dimensional domain image.

As shown, the domain converter 232 converts the domain of the sensing data received from the lidar sensor, the first camera, the radar sensor, and the second camera into two-dimensional image data 310, 320, 330, and 340. can be created.

In one embodiment, the image matching unit 234 may compare the two-dimensional image data 310, 320, 330, and 340 to detect a sensor with an input error. As shown, in the domain-converted lidar data 310, first camera data 320, and radar data 330, one vehicle (312, 322, 332) and two pedestrians (314, 324, 334) ) can be identified similarly. On the other hand, unlike the other data 310, 320, and 330, the domain-converted second camera data 340 does not identify vehicles or pedestrians due to obstacles (leaves, snow, etc.), so the data is It can be detected that an error has occurred in the input sensor.

The image matching unit 234 may determine that the similarity between the image data 310, 320, and 330 among the image data 310, 320, 330, and 340 does not exceed a threshold. On the other hand, the image matching unit 234 may classify the image data 340 as abnormal image data because the similarity between the image data 340 and the other image data 310, 320, and 330 exceeds a threshold.

Figure 4 is a diagram illustrating an example of a method for determining the current status of a driver (passenger) according to an embodiment of the present disclosure.

In one embodiment, the driver (passenger) state determination method 400 determines whether the driver is drowsy or alert according to whether the driver's eyes blink from images taken of the driver's eyes and the area around the eyes, by the driving mode determination unit. It can be executed.

In one embodiment, when the driving mode determination unit determines that the driver is in an alert state 410, a warning message may be output through the warning unit and a control signal may be generated to switch to semi-autonomous driving or manual driving. Meanwhile, if the driving mode determination unit determines that the driver is drowsy (420), it may generate a control signal to switch to the emergency driving mode.

Figure 5 is a flowchart of an autonomous driving control method based on a plurality of sensors according to an embodiment of the present disclosure.

As shown, the autonomous driving control method 500 based on a plurality of sensors may begin by receiving a plurality of first sensing data from a plurality of sensors in step 510. In one embodiment, referring to FIGS. 1 to 4, the processing unit 220 uses a plurality of sensors in the sensor unit 210, that is, a camera sensor 130, a radar 140, and a lidar 150. Sensing data can be received.

Next, the domains of the first plurality of sensing data may be converted to generate a plurality of second sensing data (step 520). In one embodiment, referring to FIGS. 1 to 4, the domain conversion unit 232 converts the domain of the first sensing data received from the processing unit 220 into two-dimensional image data 310 and 320, which are the second sensing data. , 330, 340) can be generated.

In step 530, based on the comparison result of the plurality of second sensing data, the sensor generating abnormal data among the plurality of sensors may be detected. In one embodiment, referring to FIGS. 1 to 4 , the comparator 230 compares a plurality of second sensing data and detects a sensor generating abnormal data among the plurality of sensors based on the comparison result.

Finally, when an abnormal sensor is detected, a warning message can be output (540). In one embodiment, referring to FIGS. 1 to 4, when an abnormal sensor is detected by the comparison unit 230, the warning unit 242 may output a warning message. For example, the warning message may include sound, vibration, or It may be a warning message by light.

Figure 6 is a flowchart of an autonomous driving control method based on a plurality of sensors according to another embodiment of the present disclosure.

As shown, the autonomous driving control method 600 based on a plurality of sensors may begin with a step 610 of receiving a plurality of first sensing data from a plurality of sensors.

Next, converting the domain of the plurality of first sensing data to generate a plurality of second sensing data (620). As a result of comparing the plurality of second sensing data with the sensing data, the sensing data with the lowest similarity to other sensing data is selected. Step 630 of detecting a sensor generating data may be performed.

In step 640, if the similarity difference between the sensed data exceeds the threshold, a warning message is output in step 650, an emergency driving path is created, and the vehicle can move along the emergency driving path (660).

Figure 7 is a flowchart of an autonomous driving control method based on a plurality of sensors according to another embodiment of the present disclosure.

As shown, the autonomous driving control method 700 based on a plurality of sensors may begin with a step 710 of receiving a plurality of first sensing data from a plurality of sensors.

Next, converting the domain of the plurality of first sensing data into image data of a two-dimensional domain to generate a plurality of second sensing data (720), and as a result of comparing the plurality of second sensing data with the sensing data, another A step 730 of detecting a sensor that generates sensing data with the lowest similarity to the sensing data may be performed.

In step 740, if it is determined that the difference in data similarity exceeds the threshold, a message in step 750 that outputs a warning message may be executed.

Additionally, in step 760, if it is determined that the driver is in a driving state (for example, in an alert state), step 770 of switching to the semi-autonomous driving mode or manual driving mode may be executed. On the other hand, if it is determined in step 760 that the driver is unable to drive (eg, drowsy), step 780 of creating an emergency driving path and moving along the emergency driving path may be performed.

The above-described method may be provided as a computer program stored in a computer-readable recording medium for execution on a computer. The medium may continuously store a computer-executable program, or may temporarily store it for execution or download. In addition, the medium may be a variety of recording or storage means in the form of a single or several pieces of hardware combined. It is not limited to a medium directly connected to a computer system and may be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And there may be something configured to store program instructions, including ROM, RAM, flash memory, etc. Additionally, examples of other media include recording or storage media managed by app stores that distribute applications, sites or servers that supply or distribute various other software, etc.

The methods, operations, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques may include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logical blocks, modules, and circuits described in connection with this disclosure may be general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or It may be implemented or performed as any combination of those designed to perform the functions described in. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

For firmware and/or software implementations, techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), and PROM ( on computer-readable media such as programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. It may also be implemented as stored instructions. Instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

Although the above-described embodiments have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the disclosure is not limited thereto and may also be implemented in conjunction with any computing environment, such as a network or distributed computing environment. . Furthermore, aspects of the subject matter of this disclosure may be implemented in multiple processing chips or devices, and storage may be similarly effected across the multiple devices. These devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described in relation to some embodiments in the specification, it should be understood that various modifications and changes may be made without departing from the scope of the present disclosure as understood by those skilled in the art. something to do. Additionally, such modifications and changes should be considered to fall within the scope of the claims appended hereto.

110: autonomous vehicle 120: module
130: Camera 140: Radar
150: Lidar 160: Obstacle
200: Autonomous driving control system 210: Sensor unit
220: Processing unit 222: Sensor bus interface
224: calculation unit 226: driving control unit
228: Perimeter monitoring unit 230: Comparison unit
232: Domain conversion unit 234: Image matching unit
236: memory 240: abnormal response unit
242: warning unit 244: driving mode decision unit
246: Emergency driving path creation unit 310: LiDAR data
320: first camera data 330: radar data
340: Second camera data 400: Status identification step

Claims (10)

In an autonomous driving control method based on a plurality of sensor data, executed by at least one processor,
Receiving a plurality of first sensing data from a plurality of sensors;
converting domains of the plurality of first sensing data to generate a plurality of second sensing data;
Detecting a sensor generating abnormal sensing data among the plurality of sensors based on a comparison result of the plurality of second sensing data; and
Including, outputting a warning message when a sensor generating the abnormal sensing data is detected,
The step of generating a plurality of second sensing data by converting the domain of the plurality of first sensing data,
Converting the plurality of first sensing data into two-dimensional domain image data to generate the plurality of second sensing data,
The step of detecting a sensor generating abnormal sensing data among the plurality of sensors based on a comparison result of the plurality of second sensing data includes:
Detecting a sensor that generates sensing data having the lowest similarity with other sensing data among the plurality of second sensing data; and
Comprising: determining whether a difference in similarity with other sensing data among the plurality of second sensing data of the sensor detected in the step of detecting the sensor generating sensing data with the lowest similarity exceeds a threshold; , Autonomous driving control method. delete delete According to paragraph 1,
When a sensor generating abnormal sensing data is detected, the step of outputting a warning message is:
An autonomous driving control method comprising outputting a warning message requesting confirmation whether the abnormal sensing data is due to environmental factors of the autonomous vehicle.
According to paragraph 1,
An autonomous driving control method further comprising switching the driving mode of the autonomous vehicle to a semi-autonomous driving mode or a manual driving mode when a sensor generating the abnormal sensing data is detected.
According to paragraph 1,
An autonomous driving control method further comprising controlling the autonomous vehicle to travel on an emergency driving path when a sensor generating the abnormal sensing data is detected.
According to paragraph 1,
The plurality of sensors include at least one of a radar sensor, a LiDAR sensor, an infrared sensor, and a camera.
delete A computer program stored in a computer-readable recording medium for executing the method according to any one of claims 1, 4, 5, 6, and 7 on a computer. delete

Images