KR102333765B1 - Autonomous drive system and vehicle - Google Patents

Abstract

The present invention provides an object detection apparatus including a plurality of sensors; and determining whether each of the plurality of sensors is faulty, determining whether autonomous driving is possible based on the determination of whether or not there is a failure, and controlling the vehicle to stop on the shoulder based on the determination of whether autonomous driving is possible It relates to an autonomous driving system including a processor.

Description

Autonomous drive system and vehicle

The present invention relates to an autonomous driving system.

A vehicle is a device that moves a user in a desired direction. A typical example is a car.

Meanwhile, for the convenience of a user using a vehicle, various sensors and electronic devices are being provided. In particular, research on an advanced driver assistance system (ADAS) is being actively conducted for user's driving convenience. Furthermore, the development of autonomous vehicles (Autonomous Vehicle) is being actively made.

Currently, in the market, vehicles that have only partially implemented autonomous driving functions are being sold. To realize a fully autonomous vehicle, many vehicle developers are working on it, and some progress is being made.

On the other hand, a plurality of sensors are used in an autonomous vehicle. When an autonomous vehicle is used, there is a risk that a sensor, which is a certain electronic device, may also malfunction.

In particular, when a malfunction occurs in a sensor during autonomous driving, an accident may occur if countermeasures are not prepared in advance.

SUMMARY OF THE INVENTION An object of the present invention is to provide an autonomous driving system in preparation for sensor failure in order to solve the above problems.

Another object of the present invention is to provide a vehicle including the autonomous driving system.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, an autonomous driving system according to an embodiment of the present invention includes an object detecting apparatus including a plurality of sensors; and determining whether any one of the plurality of sensors is faulty, determining whether autonomous driving is possible based on the determination of whether or not the fault is present, and based on the determination of whether autonomous driving is possible, the vehicle is stopped on the shoulder It includes a processor that controls so that it becomes possible.

The details of other embodiments are included in the detailed description and drawings.

According to an embodiment of the present invention, there are one or more of the following effects.

First, even when a malfunction of the sensor occurs while driving, there is an effect that can closely respond.

Second, there is an effect of preventing accidents by switching to manual driving or stopping on the shoulder in case of sensor failure.

Third, there is an effect of preventing malfunction of autonomous driving by limiting the autonomous driving level and deviated from the autonomous driving lane in the event of a sensor failure.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.
2 is a view of a vehicle according to an embodiment of the present invention viewed from various angles from the outside.
3 to 4 are views illustrating the interior of a vehicle according to an embodiment of the present invention.
5 to 6 are diagrams referenced to describe an object according to an embodiment of the present invention.
7 is a block diagram referenced for explaining a vehicle according to an embodiment of the present invention.
8 is a control block diagram of an autonomous driving system according to an embodiment of the present invention.
9 is a flowchart of an autonomous driving system according to an embodiment of the present invention.
10 is a diagram referenced for explaining an operation of determining whether a sensor has failed according to an embodiment of the present invention.
11 is a diagram referenced for explaining an operation of determining whether a sensor has failed according to an embodiment of the present invention.
12 is a diagram referenced for explaining an operation of an autonomous driving system when it is determined that a sensor has failed according to an embodiment of the present invention.
13 is a diagram referenced to explain a stop control operation according to an embodiment of the present invention.
14 is a diagram referenced to describe a user interface device according to an embodiment of the present invention.
15 is a diagram referenced for explaining an operation of an autonomous driving system according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "module" and "part" for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have a meaning or role distinct from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as "comprises" or "have" are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The vehicle described in this specification may be a concept including an automobile and a motorcycle. Hereinafter, the vehicle will be mainly described with respect to the vehicle.

The vehicle described herein may be a concept including both an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

In the following description, the left side of the vehicle means the left side in the driving direction of the vehicle, and the right side of the vehicle means the right side in the driving direction of the vehicle.

1 is a view showing the exterior of a vehicle according to an embodiment of the present invention.

2 is a view of a vehicle according to an embodiment of the present invention viewed from various angles from the outside.

3 to 4 are views illustrating the interior of a vehicle according to an embodiment of the present invention.

5 to 6 are diagrams referenced to describe an object according to an embodiment of the present invention.

7 is a block diagram referenced for explaining a vehicle according to an embodiment of the present invention.

1 to 7 , the vehicle 100 may include wheels rotated by a power source and a steering input device 510 for controlling the traveling direction of the vehicle 100 .

The vehicle 100 may be an autonomous driving vehicle.

The vehicle 100 may be switched to an autonomous driving mode or a manual mode based on a user input.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on a user input received through the user interface device 200 .

The vehicle 100 may be switched to an autonomous driving mode or a manual mode based on driving situation information.

The driving situation information may include at least one of object information outside the vehicle, navigation information, and vehicle state information.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the driving situation information generated by the object detection apparatus 300 .

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on driving situation information received through the communication device 400 .

The vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on information, data, and signals provided from an external device.

When the vehicle 100 is operated in the autonomous driving mode, the autonomous driving vehicle 100 may be operated based on the driving system 700 .

For example, the autonomous vehicle 100 may be driven based on information, data, or signals generated by the driving system 710 , the taking-out system 740 , and the parking system 750 .

When the vehicle 100 is operated in the manual mode, the autonomous driving vehicle 100 may receive a user input for driving through the driving manipulation device 500 . Based on a user input received through the driving manipulation device 500 , the vehicle 100 may be driven.

The overall length refers to the length from the front part to the rear part of the vehicle 100 , the width refers to the width of the vehicle 100 , and the height refers to the length from the lower part of the wheel to the roof. In the following description, the overall length direction (L) is the standard direction for measuring the overall length of the vehicle 100 , the full width direction (W) is the standard direction for measuring the overall width of the vehicle 100 , and the total height direction (H) is the vehicle (100) may mean a direction that is a reference for measuring the total height.

As illustrated in FIG. 7 , the vehicle 100 includes a user interface device 200 , an object detection device 300 , a communication device 400 , a driving manipulation device 500 , a vehicle driving device 600 , and a driving system. 700 , a navigation system 770 , a sensing unit 120 , an interface unit 130 , a memory 140 , a control unit 170 , and a power supply unit 190 .

Depending on the embodiment, the vehicle 100 may further include other components in addition to the components described herein, or may not include some of the components described herein.

The user interface device 200 is a device for communication between the vehicle 100 and a user. The user interface device 200 may receive a user input and provide information generated in the vehicle 100 to the user. The vehicle 100 may implement User Interfaces (UIs) or User Experiences (UXs) through the user interface device 200 .

The user interface device 200 may include an input unit 210 , an internal camera 220 , a biometric sensor 230 , an output unit 250 , and a processor 270 .

According to an embodiment, the user interface device 200 may further include other components in addition to the described components, or may not include some of the described components.

The input unit 210 is for receiving information from a user, and the data collected by the input unit 210 may be analyzed by the processor 270 and processed as a user's control command.

The input unit 210 may be disposed inside the vehicle. For example, the input unit 210 may include one region of a steering wheel, one region of an instrument panel, one region of a seat, one region of each pillar, and a door. One area of the (door), one area of the center console, one area of the head lining (head lining), one area of the sun visor, one area of the windshield (windshield) or the window (window) It may be disposed in one area or the like.

The input unit 210 may include a voice input unit 211 , a gesture input unit 212 , a touch input unit 213 , and a mechanical input unit 214 .

The voice input unit 211 may convert the user's voice input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170 .

The voice input unit 211 may include one or more microphones.

The gesture input unit 212 may convert the user's gesture input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170 .

The gesture input unit 212 may include at least one of an infrared sensor and an image sensor for detecting a user's gesture input.

According to an embodiment, the gesture input unit 212 may detect a user's 3D gesture input. To this end, the gesture input unit 212 may include a light output unit that outputs a plurality of infrared lights or a plurality of image sensors.

The gesture input unit 212 may detect the user's 3D gesture input through a time of flight (TOF) method, a structured light method, or a disparity method.

The touch input unit 213 may convert a user's touch input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170 .

The touch input unit 213 may include a touch sensor for sensing a user's touch input.

According to an embodiment, the touch input unit 213 may be integrally formed with the display unit 251 to implement a touch screen. Such a touch screen may provide both an input interface and an output interface between the vehicle 100 and the user.

The mechanical input unit 214 may include at least one of a button, a dome switch, a jog wheel, and a jog switch. The electrical signal generated by the mechanical input unit 214 may be provided to the processor 270 or the control unit 170 .

The mechanical input unit 214 may be disposed on a steering wheel, a center fascia, a center console, a cockpick module, a door, and the like.

The internal camera 220 may acquire an image inside the vehicle. The processor 270 may detect the user's state based on the image inside the vehicle. The processor 270 may acquire the user's gaze information from the image inside the vehicle. The processor 270 may detect the user's gesture from the image inside the vehicle.

The biometric sensor 230 may obtain biometric information of the user. The biometric sensor 230 may include a sensor capable of obtaining the user's biometric information, and may obtain the user's fingerprint information, heart rate information, and the like, using the sensor. The biometric information may be used for user authentication.

The output unit 250 is for generating an output related to visual, auditory or tactile sense.

The output unit 250 may include at least one of a display unit 251 , a sound output unit 252 , and a haptic output unit 253 .

The display unit 251 may display graphic objects corresponding to various pieces of information.

The display unit 251 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (Flexible Display). display), a three-dimensional display (3D display), and an electronic ink display (e-ink display) may include at least one.

The display unit 251 may form a layer structure with the touch input unit 213 or be integrally formed to implement a touch screen.

The display unit 251 may be implemented as a head up display (HUD). When the display unit 251 is implemented as a HUD, the display unit 251 may include a projection module to output information through an image projected on the windshield or window.

The display unit 251 may include a transparent display. The transparent display may be attached to a windshield or window.

The transparent display may display a predetermined screen while having predetermined transparency. Transparent display, in order to have transparency, transparent display is transparent TFEL (Thin Film Elecroluminescent), transparent OLED (Organic Light-Emitting Diode), transparent LCD (Liquid Crystal Display), transmissive transparent display, transparent LED (Light Emitting Diode) display may include at least one of The transparency of the transparent display can be adjusted.

Meanwhile, the user interface apparatus 200 may include a plurality of display units 251a to 251g.

The display unit 251 includes one area of the steering wheel, one area 251a, 251b, and 251e of the instrument panel, one area 251d of the seat, one area 251f of each pillar, and one area of the door ( 251g), one area of the center console, one area of the head lining, one area of the sun visor, or one area 251c of the windshield and one area 251h of the window.

The sound output unit 252 converts an electrical signal provided from the processor 270 or the control unit 170 into an audio signal and outputs the converted signal. To this end, the sound output unit 252 may include one or more speakers.

The haptic output unit 253 generates a tactile output. For example, the haptic output unit 253 may vibrate the steering wheel, the seat belt, and the seats 110FL, 110FR, 110RL, and 110RR so that the user can recognize the output.

The processor 270 may control the overall operation of each unit of the user interface device 200 .

According to an embodiment, the user interface apparatus 200 may include a plurality of processors 270 or may not include the processors 270 .

When the user interface device 200 does not include the processor 270 , the user interface device 200 may be operated under the control of a processor or controller 170 of another device in the vehicle 100 .

Meanwhile, the user interface device 200 may be referred to as a vehicle display device.

The user interface device 200 may be operated under the control of the controller 170 .

The object detecting apparatus 300 is an apparatus for detecting an object located outside the vehicle 100 . The object detection apparatus 300 may generate object information based on the sensed data.

The object information may include information on the existence of an object, location information of the object, distance information between the vehicle 100 and the object, and relative speed information between the vehicle 100 and the object.

The object may be various objects related to the operation of the vehicle 100 .

5 to 6 , the object O includes a lane OB10, another vehicle OB11, a pedestrian OB12, a two-wheeled vehicle OB13, traffic signals OB14, OB15, light, road, structure, This may include speed bumps, features, animals, and the like.

The lane OB10 may be a driving lane, a lane next to the driving lane, or a lane in which vehicles that are opposed to each other travel. The lane OB10 may be a concept including left and right lines forming a lane. The lane may be a concept including an intersection.

The other vehicle OB11 may be a vehicle running in the vicinity of the vehicle 100 . The other vehicle may be a vehicle located within a predetermined distance from the vehicle 100 . For example, the other vehicle OB11 may be a vehicle preceding or following the vehicle 100 .

The pedestrian OB12 may be a person located in the vicinity of the vehicle 100 . The pedestrian OB12 may be a person located within a predetermined distance from the vehicle 100 . For example, the pedestrian OB12 may be a person located on a sidewalk or a roadway.

The two-wheeled vehicle OB13 may refer to a vehicle positioned around the vehicle 100 and moving using two wheels. The two-wheeled vehicle OB13 may be a vehicle having two wheels positioned within a predetermined distance from the vehicle 100 . For example, the two-wheeled vehicle OB13 may be a motorcycle or a bicycle located on a sidewalk or roadway.

The traffic signal may include a traffic light OB15, a traffic sign OB14, and a pattern or text drawn on a road surface.

The light may be light generated from a lamp provided in another vehicle. The light, can be the light generated from the street lamp. The light may be sunlight.

The road may include a road surface, a curve, an uphill slope, a downhill slope, and the like.

The structure may be an object located around a road and fixed to the ground. For example, the structure may include a street lamp, a street tree, a building, a power pole, a traffic light, a bridge, a curb, and a wall surface.

Features may include mountains, hills, and the like.

Meanwhile, the object may be classified into a moving object and a still object. For example, the moving object may be a concept including another moving vehicle and a moving pedestrian. For example, the stationary object may be a concept including a traffic signal, a road, a structure, another stopped vehicle, and a stationary pedestrian.

The object detection apparatus 300 may include a camera 310 , a radar 320 , a lidar 330 , an ultrasonic sensor 340 , an infrared sensor 350 , and a processor 370 .

According to an embodiment, the object detecting apparatus 300 may further include other components in addition to the described components, or may not include some of the described components.

The camera 310 may be located at an appropriate place outside the vehicle in order to acquire an image outside the vehicle. The camera 310 may be a mono camera, a stereo camera 310a, an AVM (Around View Monitoring) camera 310b, or a 360 degree camera.

The camera 310 may obtain position information of an object, distance information from an object, or relative speed information with an object by using various image processing algorithms.

For example, the camera 310 may acquire distance information and relative velocity information from an object based on a change in the size of the object over time from the acquired image.

For example, the camera 310 may obtain distance information and relative speed information from an object through a pinhole model, road surface profiling, or the like.

For example, the camera 310 may acquire distance information and relative velocity information from an object based on disparity information from the stereo image acquired by the stereo camera 310a.

For example, the camera 310 may be disposed adjacent to the front windshield in the interior of the vehicle to acquire an image of the front of the vehicle. Alternatively, the camera 310 may be disposed around the front bumper or the radiator grill.

For example, the camera 310 may be disposed adjacent to the rear glass in the interior of the vehicle in order to acquire an image of the rear of the vehicle. Alternatively, the camera 310 may be disposed around a rear bumper, a trunk, or a tailgate.

For example, the camera 310 may be disposed adjacent to at least one of the side windows in the interior of the vehicle in order to acquire an image of the side of the vehicle. Alternatively, the camera 310 may be disposed around a side mirror, a fender, or a door.

The camera 310 may provide the acquired image to the processor 370 .

The radar 320 may include an electromagnetic wave transmitter and a receiver. The radar 320 may be implemented in a pulse radar method or a continuous wave radar method in view of a radio wave emission principle. The radar 320 may be implemented in a frequency modulated continuous wave (FMCW) method or a frequency shift keyong (FSK) method according to a signal waveform among continuous wave radar methods.

The radar 320 detects an object based on a time of flight (TOF) method or a phase-shift method using an electromagnetic wave as a medium, and a position of the detected object, a distance from the detected object, and a relative speed can be detected.

The radar 320 may be disposed at an appropriate location outside the vehicle to detect an object located in front, rear or side of the vehicle.

The lidar 330 may include a laser transmitter and a receiver. The lidar 330 may be implemented in a time of flight (TOF) method or a phase-shift method.

The lidar 330 may be implemented as a driven or non-driven type.

When implemented as a driving type, the lidar 330 is rotated by a motor and may detect an object around the vehicle 100 .

When implemented as a non-driving type, the lidar 330 may detect an object located within a predetermined range with respect to the vehicle 100 by light steering. The vehicle 100 may include a plurality of non-driven lidars 330 .

The lidar 330 detects an object based on a time of flight (TOF) method or a phase-shift method as a laser light medium, and determines the position of the detected object, the distance from the detected object, and Relative speed can be detected.

The lidar 330 may be disposed at an appropriate location outside the vehicle to detect an object located in the front, rear, or side of the vehicle.

The ultrasonic sensor 340 may include an ultrasonic transmitter and a receiver. The ultrasound sensor 340 may detect an object based on ultrasound, and detect a position of the detected object, a distance from the detected object, and a relative speed.

The ultrasonic sensor 340 may be disposed at an appropriate location outside the vehicle to detect an object located in the front, rear, or side of the vehicle.

The infrared sensor 350 may include an infrared transmitter and a receiver. The infrared sensor 340 may detect an object based on infrared light, and detect a position of the detected object, a distance from the detected object, and a relative speed.

The infrared sensor 350 may be disposed at an appropriate location outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The processor 370 may control the overall operation of each unit of the object detection apparatus 300 .

The processor 370 compares data sensed by the camera 310 , the radar 320 , the lidar 330 , the ultrasonic sensor 340 , and the infrared sensor 350 with pre-stored data to detect an object or can be classified.

The processor 370 may detect and track the object based on the acquired image. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with respect to an object through an image processing algorithm.

For example, the processor 370 may acquire distance information and relative velocity information from the obtained image based on a change in object size according to time.

For example, the processor 370 may obtain distance information and relative speed information from an object through a pinhole model, road surface profiling, or the like.

For example, the processor 370 may obtain distance information and relative velocity information from an object based on disparity information in the stereo image obtained from the stereo camera 310a.

The processor 370 may detect and track the object based on the reflected electromagnetic wave that is reflected by the object and returns. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with respect to the object based on the electromagnetic wave.

The processor 370 may detect and track the object based on the reflected laser light from which the transmitted laser is reflected by the object and returned. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with respect to the object based on the laser light.

The processor 370 may detect and track the object based on the reflected ultrasound reflected back by the transmitted ultrasound. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with respect to the object based on the ultrasound.

The processor 370 may detect and track the object based on the reflected infrared light reflected back by the transmitted infrared light. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with respect to the object based on the infrared light.

According to an embodiment, the object detecting apparatus 300 may include a plurality of processors 370 or may not include the processors 370 . For example, each of the camera 310 , the radar 320 , the lidar 330 , the ultrasonic sensor 340 , and the infrared sensor 350 may individually include a processor.

When the object detection apparatus 300 does not include the processor 370 , the object detection apparatus 300 may be operated under the control of the processor or the controller 170 of the apparatus in the vehicle 100 .

The object detection apparatus 300 may be operated under the control of the controller 170 .

The communication apparatus 400 is an apparatus for performing communication with an external device. Here, the external device may be another vehicle, a mobile terminal, or a server.

The communication device 400 may include at least one of a transmit antenna, a receive antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The communication device 400 includes a short-range communication unit 410, a location information unit 420, a V2X communication unit 430, an optical communication unit 440, a broadcast transceiver 450, an Intelligent Transport Systems (ITS) communication unit 460 and a processor. (470).

According to an embodiment, the communication device 400 may further include other components in addition to the described components, or may not include some of the described components.

The short-range communication unit 410 is a unit for short-range communication. Short-range communication unit 410, Bluetooth (Bluetooth™), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association; IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless) -Fidelity), Wi-Fi Direct, and wireless USB (Wireless Universal Serial Bus) technology may be used to support short-distance communication.

The short-range communication unit 410 may form wireless area networks to perform short-range communication between the vehicle 100 and at least one external device.

The location information unit 420 is a unit for obtaining location information of the vehicle 100 . For example, the location information unit 420 may include a Global Positioning System (GPS) module or a Differential Global Positioning System (DGPS) module.

The V2X communication unit 430 is a unit for performing wireless communication with a server (V2I: Vehicle to Infra), another vehicle (V2V: Vehicle to Vehicle), or a pedestrian (V2P: Vehicle to Pedestrian). The V2X communication unit 430 may include an RF circuit capable of implementing protocols for communication with infrastructure (V2I), vehicle-to-vehicle communication (V2V), and communication with pedestrians (V2P).

The optical communication unit 440 is a unit for performing communication with an external device via light. The optical communication unit 440 may include an optical transmitter that converts an electrical signal into an optical signal to transmit to the outside, and an optical receiver that converts the received optical signal into an electrical signal.

According to an embodiment, the light transmitter may be formed to be integrated with a lamp included in the vehicle 100 .

The broadcast transceiver 450 is a unit for receiving a broadcast signal from an external broadcast management server or transmitting a broadcast signal to the broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal.

The ITS communication unit 460 may exchange information, data or signals with the transportation system. The ITS communication unit 460 may provide the obtained information and data to the transportation system. The ITS communication unit 460 may receive information, data, or signals from the transportation system. For example, the ITS communication unit 460 may receive road traffic information from the traffic system and provide it to the controller 170 . For example, the ITS communication unit 460 may receive a control signal from the traffic system and provide it to the controller 170 or a processor provided in the vehicle 100 .

The processor 470 may control the overall operation of each unit of the communication device 400 .

According to an embodiment, the communication device 400 may include a plurality of processors 470 or may not include the processors 470 .

When the communication device 400 does not include the processor 470 , the communication device 400 may be operated under the control of a processor or controller 170 of another device in the vehicle 100 .

Meanwhile, the communication device 400 may implement a vehicle display device together with the user interface device 200 . In this case, the vehicle display device may be referred to as a telematics device or an AVN (Audio Video Navigation) device.

The communication device 400 may be operated under the control of the controller 170 .

The driving operation device 500 is a device that receives a user input for driving.

In the manual mode, the vehicle 100 may be driven based on a signal provided by the driving manipulation device 500 .

The driving manipulation device 500 may include a steering input device 510 , an acceleration input device 530 , and a brake input device 570 .

The steering input device 510 may receive a driving direction input of the vehicle 100 from a user. The steering input device 510 is preferably formed in a wheel shape to enable steering input by rotation. According to an embodiment, the steering input device may be formed in the form of a touch screen, a touch pad, or a button.

The acceleration input device 530 may receive an input for acceleration of the vehicle 100 from a user. The brake input device 570 may receive an input for decelerating the vehicle 100 from a user. The acceleration input device 530 and the brake input device 570 are preferably formed in the form of pedals. According to an embodiment, the acceleration input device or the brake input device may be formed in the form of a touch screen, a touch pad, or a button.

The driving operation device 500 may be operated under the control of the controller 170 .

The vehicle driving device 600 is a device that electrically controls driving of various devices in the vehicle 100 .

The vehicle driving unit 600 may include a power train driving unit 610 , a chassis driving unit 620 , a door/window driving unit 630 , a safety device driving unit 640 , a lamp driving unit 650 , and an air conditioning driving unit 660 . can

Depending on the embodiment, the vehicle driving apparatus 600 may further include other components in addition to the described components, or may not include some of the described components.

Meanwhile, the vehicle driving apparatus 600 may include a processor. Each unit of the vehicle driving apparatus 600 may each individually include a processor.

The power train driver 610 may control the operation of the power train device.

The power train driving unit 610 may include a power source driving unit 611 and a transmission driving unit 612 .

The power source driving unit 611 may control the power source of the vehicle 100 .

For example, when a fossil fuel-based engine is a power source, the power source driving unit 611 may perform electronic control of the engine. Thereby, the output torque of an engine, etc. can be controlled. The power source driving unit 611 may adjust the engine output torque according to the control of the control unit 170 .

For example, when the electric energy-based motor is the power source, the power source driving unit 611 may control the motor. The power source driving unit 611 may adjust the rotation speed, torque, and the like of the motor under the control of the control unit 170 .

The transmission driving unit 612 may control the transmission.

The transmission driving unit 612 may adjust the state of the transmission. The transmission driving unit 612 may adjust the transmission state to forward (D), reverse (R), neutral (N), or park (P).

Meanwhile, when the engine is the power source, the transmission driving unit 612 may adjust the engagement state of the gear in the forward (D) state.

The chassis driving unit 620 may control the operation of the chassis device.

The chassis driving unit 620 may include a steering driving unit 621 , a brake driving unit 622 , and a suspension driving unit 623 .

The steering driving unit 621 may perform electronic control of a steering apparatus in the vehicle 100 . The steering driving unit 621 may change the traveling direction of the vehicle.

The brake driving unit 622 may perform electronic control of a brake apparatus in the vehicle 100 . For example, the speed of the vehicle 100 may be reduced by controlling the operation of a brake disposed on the wheel.

Meanwhile, the brake driving unit 622 may individually control each of the plurality of brakes. The brake driving unit 622 may differently control the braking force applied to the plurality of wheels.

The suspension driving unit 623 may electronically control a suspension apparatus in the vehicle 100 . For example, when there is a curve in the road surface, the suspension driving unit 623 may control the suspension device to reduce vibration of the vehicle 100 .

Meanwhile, the suspension driving unit 623 may individually control each of the plurality of suspensions.

The door/window driving unit 630 may perform electronic control of a door apparatus or a window apparatus in the vehicle 100 .

The door/window driving unit 630 may include a door driving unit 631 and a window driving unit 632 .

The door driving unit 631 may control the door device. The door driving unit 631 may control opening and closing of a plurality of doors included in the vehicle 100 . The door driving unit 631 may control opening or closing of a trunk or a tail gate. The door driving unit 631 may control opening or closing of a sunroof.

The window driving unit 632 may perform electronic control of a window apparatus. Opening or closing of a plurality of windows included in the vehicle 100 may be controlled.

The safety device driving unit 640 may perform electronic control of various safety apparatuses in the vehicle 100 .

The safety device driving unit 640 may include an airbag driving unit 641 , a seat belt driving unit 642 , and a pedestrian protection device driving unit 643 .

The airbag driving unit 641 may perform electronic control of an airbag apparatus in the vehicle 100 . For example, the airbag driver 641 may control the airbag to be deployed when a danger is detected.

The seat belt driving unit 642 may perform electronic control of a seat belt appartus in the vehicle 100 . For example, the seat belt driving unit 642 may control the occupant to be fixed to the seats 110FL, 110FR, 110RL, and 110RR using the seat belt when a danger is sensed.

The pedestrian protection device driving unit 643 may perform electronic control for the hood lift and the pedestrian airbag. For example, when detecting a collision with a pedestrian, the pedestrian protection device driving unit 643 may control to lift up the hood and deploy the pedestrian airbag.

The lamp driver 650 may electronically control various lamp apparatuses in the vehicle 100 .

The air conditioning driving unit 660 may perform electronic control of an air conditioner (air cinditioner) in the vehicle 100 . For example, when the temperature inside the vehicle is high, the air conditioning driving unit 660 may control the air conditioner to operate to supply cool air to the interior of the vehicle.

The vehicle driving apparatus 600 may include a processor. Each unit of the vehicle driving apparatus 600 may each individually include a processor.

The vehicle driving apparatus 600 may be operated under the control of the controller 170 .

The operation system 700 is a system for controlling various operations of the vehicle 100 . The driving system 700 may be operated in an autonomous driving mode.

The driving system 700 may include a driving system 710 , a vehicle taking-out system 740 , and a parking system 750 .

Depending on the embodiment, the navigation system 700 may further include other components in addition to the described components, or may not include some of the described components.

Meanwhile, the driving system 700 may include a processor. Each unit of the navigation system 700 may each individually include a processor.

Meanwhile, according to an embodiment, when the operating system 700 is implemented in software, it may be a sub-concept of the control unit 170 .

Meanwhile, according to an embodiment, the driving system 700 includes the user interface device 270 , the object detection device 300 and the communication device 400 , the driving manipulation device 500 , the vehicle driving device 600 , and the navigation system. It may be a concept including at least one of the 770 , the sensing unit 120 , and the control unit 170 .

The driving system 710 may perform driving of the vehicle 100 .

When the vehicle 100 is an autonomous driving vehicle, the driving system 710 may be referred to as an autonomous driving system.

The driving system 710 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving apparatus 600 to drive the vehicle 100 .

The driving system 710 may receive object information from the object detecting apparatus 300 , and provide a control signal to the vehicle driving apparatus 600 to drive the vehicle 100 .

The driving system 710 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving apparatus 600 to drive the vehicle 100 .

The driving system 710 includes a user interface device 270 , an object detecting device 300 and a communication device 400 , a driving manipulation device 500 , a vehicle driving device 600 , a navigation system 770 , and a sensing unit ( 120 ) and the controller 170 may include at least one of the system concepts for driving the vehicle 100 .

Such a driving system 710 may be referred to as a vehicle driving control device.

The un-parking system 740 may perform un-parking of the vehicle 100 .

The un-parking system 740 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving apparatus 600 to un-park the vehicle 100 .

The un-parking system 740 may receive the object information from the object detection apparatus 300 and provide a control signal to the vehicle driving apparatus 600 to un-park the vehicle 100 .

The un-parking system 740 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving apparatus 600 to un-park the vehicle 100 .

The vehicle taking-out system 740 includes a user interface device 270 , an object detecting device 300 and a communication device 400 , a driving manipulation device 500 , a vehicle driving device 600 , a navigation system 770 , and a sensing unit ( 120 ) and the controller 170 may include at least one of the system concept for taking out the vehicle 100 .

The un-parking system 740 may be referred to as a vehicle un-parking control device.

The parking system 750 may perform parking of the vehicle 100 .

The parking system 750 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving device 600 to park the vehicle 100 .

The parking system 750 may receive object information from the object detection apparatus 300 , and may provide a control signal to the vehicle driving apparatus 600 to park the vehicle 100 .

The parking system 750 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving apparatus 600 to park the vehicle 100 .

The parking system 750 includes a user interface device 270 , an object detection device 300 and a communication device 400 , a driving operation device 500 , a vehicle driving device 600 , a navigation system 770 , and a sensing unit ( 120) and the controller 170, including at least one, may be a system concept for performing parking of the vehicle 100.

Such a parking system 750 may be referred to as a vehicle parking control device.

The navigation system 770 may provide navigation information. The navigation information may include at least one of map information, set destination information, route information according to the destination setting, information on various objects on a route, lane information, and current location information of the vehicle.

The navigation system 770 may include a memory and a processor. The memory may store navigation information. The processor may control the operation of the navigation system 770 .

According to an embodiment, the navigation system 770 may receive information from an external device through the communication device 400 and update pre-stored information.

According to an embodiment, the navigation system 770 may be classified into sub-components of the user interface device 200 .

The sensing unit 120 may sense the state of the vehicle. The sensing unit 120 includes an inertial navigation unit (IMU) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight sensor, a heading sensor, a position module, and a vehicle. Includes forward/reverse sensor, battery sensor, fuel sensor, tire sensor, steering sensor by steering wheel rotation, vehicle interior temperature sensor, vehicle interior humidity sensor, ultrasonic sensor, ambient light sensor, accelerator pedal position sensor, brake pedal position sensor, etc. can do.

Meanwhile, an inertial navigation unit (IMU) sensor may include at least one of an acceleration sensor, a gyro sensor, and a magnetic sensor.

The sensing unit 120 may include vehicle posture information, vehicle motion information, vehicle yaw information, vehicle roll information, vehicle pitch information, vehicle collision information, vehicle direction information, and vehicle location information (GPS information). ), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle inclination information, vehicle forward/reverse information, battery information, fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, steering wheel rotation Sensing signals for an angle, external illumination of the vehicle, pressure applied to the accelerator pedal, and pressure applied to the brake pedal may be acquired.

The sensing unit 120 is, in addition, an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor. (TPS), a TDC sensor, a crank angle sensor (CAS), and the like.

The sensing unit 120 may generate vehicle state information based on the sensing data. The vehicle state information may be information generated based on data sensed by various sensors provided inside the vehicle.

For example, the vehicle state information may include vehicle posture information, vehicle speed information, vehicle inclination information, vehicle weight information, vehicle direction information, vehicle battery information, vehicle fuel information, vehicle tire pressure information, It may include vehicle steering information, vehicle indoor temperature information, vehicle indoor humidity information, pedal position information, and vehicle engine temperature information.

The interface unit 130 may serve as a passage with various types of external devices connected to the vehicle 100 . For example, the interface unit 130 may have a port connectable to the mobile terminal, and may connect to the mobile terminal through the port. In this case, the interface unit 130 may exchange data with the mobile terminal.

Meanwhile, the interface unit 130 may serve as a passage for supplying electrical energy to the connected mobile terminal. When the mobile terminal is electrically connected to the interface unit 130 , under the control of the controller 170 , the interface unit 130 may provide the electric energy supplied from the power supply unit 190 to the mobile terminal.

The memory 140 is electrically connected to the control unit 170 . The memory 140 may store basic data for the unit, control data for operation control of the unit, and input/output data. The memory 140 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware. The memory 140 may store various data for the overall operation of the vehicle 100 , such as a program for processing or controlling the controller 170 .

According to an embodiment, the memory 140 may be formed integrally with the control unit 170 or may be implemented as a sub-component of the control unit 170 .

The controller 170 may control the overall operation of each unit in the vehicle 100 . The control unit 170 may be referred to as an Electronic Control Unit (ECU).

The power supply unit 190 may supply power required for operation of each component under the control of the control unit 170 . In particular, the power supply unit 190 may receive power from a battery inside the vehicle.

Included in the vehicle 100, the one or more processors and control unit 170, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs ( field programmable gate arrays), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

8 is a control block diagram of an autonomous driving system according to an embodiment of the present invention.

Referring to FIG. 8 , the autonomous driving system 710 includes a user interface device 200 , an object detection device 300 , a communication device 400 , an interface unit 713 , a memory 714 , a processor 717 , and A power supply unit 719 may be included.

As for the user interface device 200 , the description of the user interface device 200 of FIGS. 1 to 7 may be applied.

The user interface device 200 may output content based on data, information, or signals generated or processed by the processor 717 .

For example, the user interface device 200 may output a manual driving change request signal. The user interface device 200 may receive a user input for manually switching driving.

As for the object detection apparatus 300 , the description of the object detection apparatus 300 of FIGS. 1 to 7 may be applied.

The object detection apparatus 300 may include a plurality of sensors.

For example, as described above, the object detection apparatus 300 may include a camera 310 , a radar 320 , a lidar 330 , an ultrasonic sensor 340 , and an infrared sensor 350 .

The object detecting apparatus 300 may detect one or more stopping areas based on the control of the processor 717 .

As for the communication device 400 , the description of the communication device 400 of FIGS. 1 to 7 may be applied.

The communication apparatus 400 may communicate with other devices.

For example, the communication device 400 may communicate with at least one of another vehicle and an external server.

The communication device 400 may receive data about the object from at least one of another vehicle and an external server.

The communication device 400 may transmit information on the selected stopping area to at least one of another vehicle and an external server.

The interface unit 713 may exchange information, signals, or data with other devices included in the vehicle 100 . The interface unit 713 may transmit the received information, signal, or data to the processor 717 . The interface unit 713 may transmit information, signals, or data generated or processed by the processor 717 to other devices included in the vehicle 100 . The interface unit 713 may receive information, signals, or data from other devices included in the vehicle 100 .

The interface unit 713 may receive driving condition information.

The memory 714 is electrically connected to the processor 717 . The memory 714 may store basic data for the unit, control data for operation control of the unit, and input/output data. The memory 714 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, and the like in terms of hardware. The memory 714 may store various data for the overall operation of the autonomous driving system 710 , such as a program for processing or controlling the processor 717 .

According to an embodiment, the memory 714 may be integrally formed with the processor 717 or implemented as a sub-component of the processor 717 .

The processor 717 may be electrically connected to each unit of the autonomous driving system 710 .

The processor 717 may control the overall operation of each unit of the autonomous driving system 710 .

The processor 717 may determine whether each of the plurality of sensors is faulty.

The processor 717 may determine whether any one of the plurality of sensors is faulty based on a plurality of data for one object that are respectively generated by the plurality of sensors.

Each of the plurality of sensors may generate respective sensing data for an object around the vehicle.

The processor 717 may compare a plurality of sensing data generated by a plurality of sensors with each other.

The processor 717 may determine whether any one of the plurality of sensors has failed based on the comparison result of the plurality of sensing data.

The processor 717, based on at least one of position data of the object, the path data of the object, the relative distance data between the object and the vehicle, and the relative speed data between the object and the vehicle, respectively generated by the plurality of sensors, It can be determined whether any one of the sensors is faulty.

The plurality of sensors may generate position data of the object, path data of the object, relative distance data between the object and the vehicle, and relative speed data between the object and the vehicle, respectively.

The processor 717 may compare object position data, object path data, object-to-vehicle relative distance data, and object-to-vehicle relative speed data generated by a plurality of sensors for each sensor.

The processor 717 may determine whether any one of the plurality of sensors has failed based on the comparison result.

The processor 717 may determine whether any one of the plurality of sensors has failed based on data received through the communication device 400 .

The processor 717 compares the plurality of data for one object generated by the plurality of sensors and the data received through the communication device 400 with each other to determine whether any one of the plurality of sensors has a failure. can

The processor 717 may receive sensing data of another vehicle for an object sensed by a plurality of sensors through the communication device 400 .

The processor 717 may compare the sensed data of the object generated by the plurality of sensors with the received sensed data of another vehicle.

The processor 717 may determine whether any one of the plurality of sensors has failed based on the comparison result.

The processor 717 may determine whether any one of the plurality of sensors has failed at a preset period.

The processor 717 may change a preset period based on the driving situation information.

For example, when the vehicle 100 is driven in a situation in which the probability of occurrence of an accident is high, the processor 717 may change the preset period more quickly based on the driving situation information.

For example, the processor 717 may change a preset period based on the driving route information or the driving section information. For example, when the vehicle 100 is scheduled to be driven in an accident-prone section, the preset cycle may be changed more quickly.

For example, the processor 717 may change a preset period based on driving time information. For example, when the vehicle 100 is driven at night, the preset cycle may be changed faster than when the vehicle 100 is driven during the day.

The processor 717 may determine whether autonomous driving is possible based on the determination of whether or not there is a failure.

For example, the processor 717 may determine that autonomous driving is possible when a rear detection sensor fails while driving on a highway.

For example, the processor 717 may determine that autonomous driving is possible when a side detection sensor fails while driving on a highway.

For example, the processor 717 may determine that autonomous driving is possible when a short-distance forward detection sensor fails while driving on a highway.

For example, the processor 717 may determine that autonomous driving is impossible when a long-distance forward detection sensor fails while driving on a high-speed road.

For example, the processor 717 may determine that autonomous driving is possible when a long-distance forward detection sensor fails while driving on a city road.

For example, the processor 717 may determine that autonomous driving is possible when a rear detection sensor fails while driving on a city road.

For example, the processor 717 may determine that autonomous driving is impossible when the short-distance forward detection sensor fails while driving on a city road.

For example, the processor 717 may determine that autonomous driving is impossible when a side detection sensor fails while driving on a city road.

If it is determined that autonomous driving is possible despite the failure of the sensor, the processor 717 may change a preset driving route.

The first path may be a path in which sensing data based on a camera is essential.

If, before entering the first route, it is determined that the camera is broken and it is determined that autonomous driving is possible, the processor 717 may change the route from the first route to the second route.

If it is determined that autonomous driving is impossible due to a malfunction of any one of the plurality of sensors while the vehicle 100 is driving on the autonomous driving road, the processor 717 is configured to: (eg, a manual driving road) can be controlled to move.

When it is determined that a failure has occurred in any one of the plurality of sensors, the processor 717 may output a manual driving change request signal through the user interface device 200 .

When the first user input signal is received through the user interface device, the processor 717 may control the vehicle 100 to be switched to the manual driving state.

The first user input signal may be a signal based on a user input for converting the user from the autonomous driving state to the manual driving state.

The processor 717 may control the vehicle 100 to stop on the shoulder based on the determination of whether autonomous driving is possible.

A shoulder may be defined as, on a road, the outermost area.

For example, the shoulder may be a lane dividing a road or an area inside a guard rail. That is, the shoulder may be a lane dividing a road or a left area of a guard rail based on a vehicle driving direction.

Specifically, when it is determined that autonomous driving is impossible due to a sensor failure, the processor 717 may control the vehicle 100 to stop on the shoulder.

The processor 717 may control the object detecting apparatus 200 to detect a stopable area on the road.

The object detection apparatus 200 may detect a stopable area. The object detecting apparatus 200 may detect a stopable area based on a sensing operation of a sensor other than a malfunctioning sensor.

The processor 717 may output information on the stopable area through the user interface device 200 .

When the second user input signal is received through the user interface device 200 , the processor 717 may select a stopping area from among one or more stopping possible areas.

The second user input signal may be a signal based on a user input for selecting a stopping area of the vehicle 100 from among one or more stopable areas.

The processor 717 may control the vehicle 100 to stop in the selected stopping area based on the second user input signal.

In this case, the processor 717 may control to move at a speed based on the driving situation.

The processor 717 may transmit information on the selected stopping area to at least one of another vehicle and an external server.

For example, the processor 717 may prevent an accident by transmitting information on the selected stopping area to another vehicle following.

For example, the processor 717 may transmit information on the selected stopping area to the traffic management server so that the vehicle 100 can be towed.

Meanwhile, the processor 717 may determine whether to operate the vehicle driving assistance apparatus based on information on the type of the malfunctioning sensor among the plurality of sensors.

The processor 717 may limit the function of the vehicle driving assistance apparatus based on information on the type of the malfunctioning sensor among the plurality of sensors.

For example, when it is determined that the front camera is malfunctioning, the processor 717 may limit the function of the vehicle driving assistance apparatus (eg, LKA, LCA) based on lane detection. In this case, when the AVM camera and lidar are in a normal state, the processor 717 may control the vehicle to move to the repair shop at a low speed based on sensing data of the AVM camera and lidar.

For example, when it is determined that the front radar has failed, the processor 717 may limit the function of the vehicle driving assistance device (eg, ACC) based on the preceding vehicle detection.

The power supply unit 719 may supply power required for operation of each component under the control of the processor 717 . The power supply unit 719 may receive power from a battery inside the vehicle.

9 is a flowchart of an autonomous driving system according to an embodiment of the present invention.

Referring to FIG. 9 , the processor 717 may control the vehicle 100 to be driven in an autonomous driving state ( S910 ).

While the vehicle 100 is driving in the autonomous driving state, the processor 717 may determine whether each of a plurality of sensors included in the object detection apparatus 300 is faulty ( S920 ).

When it is determined that one or more of the plurality of sensors has failed, the processor 717 may output sensor failure situation information through the user interface device 200 ( S930 ).

When it is determined that one or more sensors among the plurality of sensors have failed, the processor 717 may transmit sensor failure situation information to at least one of another vehicle and a server through the communication device 400 (S930).

The processor 717 may determine whether it is possible to switch to the manual driving state based on the first user input ( S940 ).

The processor 717 may output a manual driving change request signal through the user interface device 200 .

When the first user input signal is received through the user interface device, the processor 717 may control the vehicle 100 to be switched to the manual driving state.

The first user input signal may be a signal based on a user input for converting the user from the autonomous driving state to the manual driving state.

When switching to manual driving, the processor 717 may control the driving to be driven based on a user input received through the driving manipulation device 500 ( S945 ).

When the first user input signal is not received, the processor 717 may determine whether autonomous driving is possible (S947).

When it is determined that autonomous driving is possible, the processor 717 may adjust the autonomous driving level ( S950 ).

For example, the processor 717 may adjust the autonomous driving level by lowering the maximum possible driving speed.

For example, the processor 717 may adjust the autonomous driving level to enable autonomous driving only on highways.

For example, the processor 717 may adjust the autonomous driving level to enable autonomous driving only on a road without traffic lights.

For example, the processor 717 may adjust the autonomous driving level to enable autonomous driving only in a state in which the user gazes forward.

For example, the processor 717 may adjust the autonomous driving level so that autonomous driving is possible only when the user's hand is positioned on the steering wheel.

When it is determined that autonomous driving is impossible, the processor 717 may determine whether a shoulder stop is possible (S960).

The processor 717 may control the object detecting apparatus 200 to detect a stopable area on the shoulder of the road.

The object detection apparatus 200 may detect a stopable area on the shoulder of a road. The object detecting apparatus 200 may detect a stopable area based on a sensing operation of a sensor other than a malfunctioning sensor.

When one or more stopable areas are detected, the processor 717 may determine that a shoulder stop is possible.

The processor 717 may output information on the stopable area through the user interface device 200 .

When the second user input signal is received through the user interface device 200 , the processor 717 may select a stopping area from among one or more stopping possible areas.

The second user input signal may be a signal based on a user input for selecting a stopping area of the vehicle 100 from among one or more stopable areas.

The processor 717 may control the vehicle 100 to be stopped in the selected stopping area based on the second user input signal (S970).

In this case, the processor 717 may control to move at a speed based on the driving situation.

The processor 717 may transmit information on the selected stopping area to at least one of another vehicle and an external server (S980).

In step S960 , if one or more stopable areas are not detected, the processor 717 may control the vehicle 100 to emergency run ( S965 ).

For example, the processor 717 may control the vehicle to travel at a low speed without changing a lane after moving to the outermost lane. In this case, the processor 717 may control the emergency lamp to be turned on.

10 is a diagram referenced for explaining an operation of determining whether a sensor has failed according to an embodiment of the present invention.

The processor 717 may determine whether any one of the plurality of sensors is faulty based on a plurality of data for one object 1000 that are respectively generated by the plurality of sensors.

The processor 717, based on at least one of position data of the object, the path data of the object, the relative distance data between the object and the vehicle, and the relative speed data between the object and the vehicle, respectively generated by the plurality of sensors, It can be determined whether any one of the sensors is faulty.

As illustrated in FIG. 10A , the camera 310 may generate relative distance data and relative speed data between the vehicle 100 and another vehicle 1000 that precedes it.

The radar 320 may generate relative distance data and relative speed data between the vehicle 100 and the preceding other vehicle 1000 .

The lidar 330 may generate relative distance data and relative speed data between the vehicle 100 and the preceding other vehicle 1000 .

The ultrasonic sensor 340 may generate relative distance data and relative speed data between the vehicle 100 and the preceding other vehicle 1000 .

The infrared sensor 350 may generate relative distance data and relative speed data between the vehicle 100 and the preceding other vehicle 1000 .

The processor 717 may compare relative distance data generated by each of the camera 310 , the radar 320 , the lidar 330 , the ultrasonic sensor 340 , and the infrared sensor 350 with each other.

The processor 717 may determine a sensor that generates data out of an error range as a result of the mutual comparison as a faulty sensor.

According to an embodiment, the processor 717 may determine the first sensor as a faulty sensor when, after repeated comparison for a preset number of times or more, the data by the first sensor is out of an error range for more than a reference number of times.

The processor 717 may compare relative velocity data generated by each of the camera 310 , the radar 320 , the lidar 330 , the ultrasonic sensor 340 , and the infrared sensor 350 with each other.

The processor 717 may determine a sensor that generates data out of an error range as a result of the mutual comparison as a faulty sensor.

According to an embodiment, the processor 717 may determine the first sensor as a faulty sensor when, after repeated comparison for a preset number of times or more, the data by the first sensor is out of an error range for more than a reference number of times.

As illustrated in FIG. 10 ( b ) , it is possible to determine whether a sensor malfunctions based on a comparison of the expected driving paths of the vehicle 100 and the other vehicle 1000 in the straight section 1001 .

The processor 717 may generate a predicted driving path of the vehicle 100 based on the sensing data of the first sensor.

The processor 717 accumulates sensing data of the first sensor that senses the surroundings (eg, lane) of the vehicle 100 for a first set time, and based on the accumulated sensing data, the straight-line section 1001 ) may generate first route information of the vehicle.

The processor 717 may generate an expected driving path of the other vehicle 1000 based on the sensing data of the second sensor.

The processor 717 accumulates the sensing data of the second sensor sensing the other vehicle 1000 and the lane for the first set time, and based on the accumulated sensing data, the other vehicle ( 1000) of the second path information may be generated.

The processor 717 may compare the first path information and the second path information with each other.

When it is determined that the intersection 1010 of the first path information and the second path information is generated, the processor 717 may determine that a failure has occurred in at least one of the first sensor and the second sensor.

11 is a diagram referenced for explaining an operation of determining whether a sensor has failed according to an embodiment of the present invention.

The processor 717 may determine whether any one of the plurality of sensors has failed based on data received through the communication device 400 .

The processor 717 compares the plurality of data for one object generated by the plurality of sensors and the data received through the communication device 400 with each other to determine whether any one of the plurality of sensors has a failure. can

11 , the processor 717 may receive sensing data for the first other vehicle 1100 through the communication device 400 .

Here, the sensing data of the first other vehicle 1100 may be generated by the first other vehicle 1100 . Alternatively, the sensing data of the first other vehicle 1100 may be generated by the second other vehicle 1110 .

The sensing data of the first other vehicle 1100 includes location data of the first other vehicle 1100 , route data of the first other vehicle 1100 , and a relative relationship between the first other vehicle 1100 and the vehicle 100 . It may include at least one of distance data and relative speed data between the first other vehicle 1100 and the vehicle 100 .

Each of the plurality of sensors may generate sensing data for the first other vehicle 1100 .

The processor 717 compares the sensing data of the first other vehicle 1100 generated by each of the plurality of sensors with the data received through the communication device 400 to determine whether each of the plurality of sensors fails. can do.

12 is a diagram referenced for explaining an operation of an autonomous driving system when it is determined that a sensor is malfunctioning according to an embodiment of the present invention.

Referring to FIG. 12 , when it is determined that a failure has occurred in any one of a plurality of sensors, the processor 717 may output a manual driving change request signal 1210 through the user interface device 200 .

The user interface device 200 may receive a first user input.

The processor 717 may receive a first user input signal based on the first user input.

When the first user input is received, the processor 717 may control the vehicle 100 to be switched to the manual driving state.

In this case, the processor 717 may control to drive based on a user driving manipulation input received by the driving manipulation device 500 .

13 is a diagram referenced to explain a stop control operation according to an embodiment of the present invention.

As illustrated in FIG. 13A , the processor 717 may distinguish and detect the road 1302 and the sidewalk 1301 through the object detection apparatus 300 .

According to an embodiment, the processor 717 may distinguish the road 1302 from the sidewalk 1301 further based on the navigation information.

As illustrated in FIG. 13B , the processor 717 may detect the outermost lane 1311 of the road.

The processor 717 may determine whether the vehicle can be stopped based on the type or color of the lane 1311 .

The processor 717 may determine whether the vehicle can be stopped based on whether the lane 1311 is a solid line or a dotted line, or whether it is yellow or white.

As illustrated in FIG. 13C , the processor 717 may detect stopable areas 1321 , 1322 , and 1323 .

The processor 717 may calculate a distance between the objects 1325 and 1326 based on the data generated by the object detection apparatus 300 .

The processor 717 may detect the stopable areas 1321 , 1322 , and 1323 based on distance information between the objects 1325 and 1326 .

As illustrated in (d) of FIG. 13 , the processor 717 may select any one of one or more vehicle stopping areas 1321 , 1322 , and 1323 as the stopping area 1332 .

According to an embodiment, the processor 717 may select the stopping area 1332 based on a user input.

According to an embodiment, the processor 717 may select the stopping area 1332 based on the surrounding context information.

The processor 717 may control the vehicle 100 to move to the selected stopping area 1332 .

In this case, the processor 717 may control the vehicle 100 to move at a predetermined constant speed.

In this case, the processor 717 may control the emergency lamp to be turned on.

In this case, the processor 717 may detect an obstacle 1331 that may be positioned between the vehicle 100 and the stopping area 1332 . The processor 717 may control to avoid the obstacle 1331 and move to the stopping area 1332 .

14 is a diagram referenced to describe a user interface device according to an embodiment of the present invention.

Referring to FIG. 14 , the processor 717 may output information 1410 on whether or not the sensor has failed through the display 251 of the user interface device 200 .

When the shoulder stopping operation is determined, the processor 717 may output the stopping operation information 1420 through the display 251 .

The processor 717 may output information on the vehicle stopping area 1431 through the display 251 .

The processor 717 may determine a stopping area 1432 among the stopping possible areas 1431 based on the driving situation information and control the vehicle 100 to stop in the stopping area 1432 .

The processor 717 determines a stopping area 1433 among the stopping possible areas 1431 based on a user input through the user interface device 200 , and controls the vehicle 100 to stop in the stopping area 1432 . can do.

15 is a diagram referenced for explaining an operation of an autonomous driving system according to an embodiment of the present invention.

The processor 717 determines a stopping area 1510 .

The processor 717 may transmit information about the stopping area 1510 to other nearby vehicles through the communication device 400 .

The information on the stopping area 1510 may include movement path information from the location of the vehicle 100 to the stopping area 1510 .

Specifically, the processor 717 may transmit information on the stopping area 1510 to other vehicles traveling in the driving lane of the vehicle 100 and in a lane located between the driving lane and the stopping area 1510 .

The processor 717 may transmit a signal requesting at least one of acceleration, deceleration, and lane change to the other vehicle based on the positional relationship between the vehicle 100 and the other vehicle.

The processor 717 runs on the driving lane 1501 of the vehicle 100 and on the lane 1512 located between the driving lane 1501 and the stopping area 1510 , and other vehicles preceding the vehicle 100 . In this case, it is possible to transmit a signal requesting acceleration.

The processor 717 runs on the driving lane 1501 of the vehicle 100 and on the lane 1513 positioned between the driving lane 1501 and the stopping area 1510 , and other vehicles following the vehicle 100 . In this case, a signal requesting deceleration may be transmitted.

In addition, the processor 717 is configured to run on the driving lane 1501 of the vehicle 100 and the lane 1513 located between the driving lane 1501 and the stop area 1510 , and other vehicles following the vehicle 100 . A signal for requesting driving in a lane 1511 other than between the driving lane of the vehicle 100 and the stopping area 1510 may be transmitted to vehicles.

The present invention described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that is implemented in the form of. In addition, the computer may include a processor or a control unit. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

100: vehicle
710: autonomous driving system

Claims (11)

user interface device;
an object detection device including a plurality of sensors; and
Determining whether each of the plurality of sensors is faulty,
Based on the determination of the failure, it is determined whether autonomous driving is possible,
a processor for controlling the vehicle to stop on the shoulder based on the determination of whether the autonomous driving is possible; and
The object detection device,
detecting one or more stopable areas;
The processor is
outputting information on the stopable area, through the user interface device,
When a second user input signal is received through the user interface device, the autonomous driving system controls the vehicle to stop in a selected stopping area based on the second user input signal. The method of claim 1,
The processor is
An autonomous driving system that determines whether any one of the plurality of sensors is faulty based on a plurality of data for one object, each of which is generated by the plurality of sensors. 3. The method of claim 2,
The processor is
Based on at least one of the position data of the object, the path data of the object, the relative distance data between the object and the vehicle, and the relative speed data between the object and the vehicle respectively generated by the plurality of sensors, the plurality of An autonomous driving system that determines whether any one of the sensors has failed. The method of claim 1,
Further comprising; a communication device for performing communication with another device;
The processor is
An autonomous driving system that determines whether any one of the plurality of sensors has failed based on data received through the communication device. 5. The method of claim 4,
The processor is
An autonomous driving system for determining whether any one of the plurality of sensors is faulty by comparing data for one object generated by the plurality of sensors and data received through the communication device with each other. 5. The method of claim 2 or 4,
The processor is
An autonomous driving system that determines whether any one of the plurality of sensors has failed at a preset period. The method of claim 1,
a user interface device; further comprising
The processor is
When it is determined that a failure has occurred in any one of the plurality of sensors,
Based on the determination of the failure, it is determined whether manual driving is possible,
output a manual driving change request signal through the user interface device,
When a first user input signal is received through the user interface device, the autonomous driving system controls the vehicle to be switched to a manual driving state. delete The method of claim 1,
Further comprising; a communication device for performing communication with another device;
The processor is
An autonomous driving system for transmitting information on the selected stopping area to at least one of another vehicle and an external server. The method of claim 1,
The processor is
An autonomous driving system that changes a preset driving route based on the determination of the failure. 11. The method of claim 10,
When it is determined that any one of the plurality of sensors is broken while the vehicle is driving on an autonomous driving road,
The processor is
An autonomous driving system that controls the vehicle to move from the autonomous driving road to a general driving road.

Images