KR102382110B1 - A self driving car system having virtual steering wheel and driving method of the same - Google Patents

Abstract

The present invention makes it possible to remotely control an autonomous vehicle more safely, and in this case, by differentially distributing control rights according to the driving ability of the remote controller, the autonomous driving and manual remote driving are harmonized, thereby remarkably improving stability.
A remote control assistance system for an autonomous vehicle according to an aspect of the present invention is a virtual system comprising: a first sensing unit for detecting a user's bio-signal or a user's manipulation signal; and a first communication unit for transmitting the user's manipulation signal; handle, and
and a control unit that converts the user operation signal received from the first communication unit, a steering unit that receives the operation signal converted by the control unit, and a driving unit that drives according to the signal of the steering unit.

Description

A self-driving vehicle system having a virtual steering wheel and a driving method thereof

The present invention relates to an autonomous driving vehicle system and a driving method thereof, and more particularly, to an autonomous driving vehicle system having a virtual steering wheel and a driving method thereof.

An autonomous vehicle is a vehicle that can drive automatically without human driving. Autonomous vehicles drive autonomously by recognizing their surroundings using radar, light detection and ranging (LIDAR), GPS, and cameras and specifying a destination. Unmanned vehicles that are already being put into practical use include unmanned vehicles that patrol preset routes operated by the Israeli military, and unmanned driving systems such as dump trucks operated in overseas mines and construction sites.

The first core technologies of these autonomous vehicles are the driverless vehicle system and the actual system. It is a technology that not only simulates in the laboratory, but also actually builds an unmanned vehicle system. It implements the accelerator, reducer and steering device, which are driving devices, to suit unmanned operation, and enables control using the computer, software and hardware installed in the unmanned vehicle. do.

The second core technology is to receive and process visual information using vision and sensors. As the basis of autonomous driving for unmanned operation, it is a technology that accepts image information and extracts necessary information from the image. It uses not only CCD (charge-coupled device) cameras but also ultrasonic sensors and range filters to fuse information necessary for distance and driving, so that it can avoid obstacles and cope with unexpected situations through analysis and processing.

The third core technology is the integrated control system and operation monitoring fault diagnosis system technology. This technology establishes a driving monitoring system that monitors vehicle operation and gives appropriate commands according to changing situations, and provides appropriate information to the operator by diagnosing system failures by analyzing various situations occurring from individual processors and sensors. or to perform the function of notifying an alarm.

The fourth core technology is intelligent control and intelligent operation devices. This technology is an unmanned driving technique that generates control commands based on mathematical analysis using real vehicle models. Intelligent Forward Control is a system that maintains the distance from the vehicle in front or obstacles by adjusting the speed by itself without the driver operating the pedals based on radar guide technology. When the driver inputs the distance to the vehicle in front, the long-distance radar attached to the front of the vehicle detects the position of the vehicle in front, maintains a constant speed, decelerates or accelerates, and stops completely if necessary, which is useful in weather where visibility is difficult.

The fifth applied technology is the lane departure prevention system. This is a technology that notifies the driver of an unintended departure situation by detecting the lane with a camera installed inside.

The sixth application technology is the parking assistance system. This is a system that helps the car park in reverse by adjusting the steering system when the driver finds the assist button, puts in reverse gear, and presses the brake pedal. By automatically guiding the vehicle from the departure point to the parking space based on infrastructure as well as vehicle-mounted sensors, it saves unnecessary time and energy when parking, thereby minimizing cost and environmental pollution.

The seventh application technology is the automatic parking system. This is a technology in which the driver stops the car in front of the parking lot, turns off the engine, gets off, and presses the remote control lock switch twice in succession. am.

The eighth application technology is a blind spot information guidance system. It is used to change lanes by checking obstacles and vehicles on both sides in complex road conditions, as sensors mounted on both sides of the car determine whether there is another vehicle in the blind spot that is not visible through the side mirrors and warn the driver. .

The biggest advantage of driverless cars is that they help fuel efficiency by avoiding repeated stops by making the control device more even because the driving speed and traffic management data match. will be. In addition, it has the advantages of solving fatigue caused by long-term driving, greatly reducing the risk of traffic accidents, speeding up road traffic flow, and reducing traffic congestion.

Car accidents are mainly caused by driver error. When a person is driving, he is distracted by external things other than driving. In addition, humans have physical limitations such as drowsiness, eyesight, and reaction time. On the other hand, driverless cars have a 360-degree field of vision and have the ability to transcend the physical limitations of humans, such as being able to see well at night with special equipment such as radar, reducing the probability of an accident.

Driverless cars will also reduce traffic congestion, increasing the amount of time you don't need to drive. In addition, driverless cars can reduce the amount of time it takes to find and park a car, and because they can better detect the presence of other cars, they can drive while keeping themselves closer to other vehicles, which reduces drag and reduces drag. work is possible In other words, reducing air resistance is associated with reducing fuel consumption.

However, if more than 80% of the cars driving on the road are driverless cars, and all of them are autonomously driving, it is not currently legally determined who should be held responsible if an accident occurs. In addition, although driving based on GPS, if internet access is possible, hacking becomes possible, and there is a risk that hackers can control it at will, and ethical issues are also a big problem. Therefore, even when entering the stage of perfect autonomous driving, there may be cases in which human judgment and human driving control are required to some extent.

[Prior art literature]

Korean Patent Registration No. 10-1869340 (2018. 06. 21.)

The present invention is to solve the problems of the prior art, and an object of the present invention is to control an autonomous driving vehicle through a virtual steering wheel to harmoniously process autonomous driving and direct human manipulation. A vehicle system and a driving method thereof are provided.

A remote control assistance system for an autonomous vehicle according to an aspect of the present invention is a virtual system comprising: a first sensing unit that detects a user's bio-signal or a user's manipulation signal; and a first communication unit that transmits the user's manipulation signal; handle, and

An autonomous driving vehicle comprising: a control unit that converts the user operation signal received from the first communication unit; a steering unit that receives the operation signal converted by the control unit; include

In this case, the first detection unit may be configured as a biosensor or a camera to detect the user's biometric information to perform user authentication.

In addition, the first detection unit may be made of any one of an acceleration sensor, a gyro sensor, and a gravity sensor to detect a physical rotation value of the virtual handle unit applied by the user.

In addition, the control unit includes a remote signal conversion module that converts the rotation value detected by the first sensing unit to transmit to the steering unit, and the original autonomous driving control state and the user even while the user controls the autonomous vehicle using the virtual handle unit. It may include at least one of a driving monitoring module for judging the control state of the driver and an autonomous driving intervention module for intervening autonomous driving in the user's control state.

In addition, the remote signal conversion module may convert the user manipulation signal of the virtual handle unit in comparison with the left and right rotation of the steering unit (handle) of the autonomous vehicle based on the 180 degree rotation of the virtual handle unit left and right.

In addition, when the central state information of the virtual handle unit is output from the screen unit and the user can start the control when the virtual handle unit belongs to the central state.

In addition, the autonomous driving intervention module may increase the amount of autonomous driving intervention at the time when control by the user starts and gradually decrease the amount of autonomous driving intervention.

In addition, the autonomous driving intervention module intervenes when the driving direction or speed determined by the driving monitoring module and the driving direction or speed controlled by the user are greater than a set value within a set time to recover part or all of the control right of the autonomous driving vehicle can make it

On the other hand, the driving method of the autonomous driving vehicle according to an aspect of the present invention includes the steps of: a user requesting to switch control of the autonomous driving vehicle; , a step in which the user starts controlling the vehicle, but the entire control right of the autonomous vehicle is not transferred, the user determines a route such as controlling the vehicle in comparison to the control of the autonomous vehicle, and the control right of the autonomous vehicle and transferring the whole to the user.

In this case, when it is determined that there is a risk of a collision after the step in which the entire control right of the autonomous driving vehicle is transferred to the user, the method may further include the step of intervening in the driving of the autonomous driving vehicle.

The present invention allows the user terminal to function as a virtual steering wheel to easily determine whether the user is suitable for driving, determine whether the control of the virtual steering wheel is consistent, and finally transfer the control right to the virtual steering wheel so that the vehicle can be operated. Complementarily combines autonomous driving with human manual driving.

In addition, the present invention further maximizes stability because the autonomous driving path is always compared and determined together even when a person is driving, so that when a momentary collision possibility is seen, the driving control right returns to the vehicle momentarily.

1 is a conceptual diagram of an autonomous vehicle having a virtual steering wheel according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a state in which a control right of a vehicle is received by using a virtual steering wheel in an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention.
3 is a diagram illustrating a state in which user authentication is performed using a virtual steering wheel in an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention.
4 is a diagram illustrating an example of driving a vehicle using a virtual steering wheel in an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention.
5 is a block diagram illustrating an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention.
FIG. 6 is a configuration diagram illustrating the first sensing unit of FIG. 5 in more detail.
FIG. 7 is a configuration diagram illustrating the second sensing unit of FIG. 5 in more detail.
FIG. 8 is a configuration diagram illustrating the control unit of FIG. 5 in more detail.
9 is a flowchart illustrating a driving method of an autonomous vehicle having a virtual steering wheel according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, a remote control assistance system for an autonomous vehicle according to an embodiment of the present invention will be described. 1 is a conceptual diagram of an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention, and FIG. 2 is a vehicle using a virtual steering wheel in the autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention. It is a view showing a state of receiving a control right, and FIG. 3 is a view showing a state in which user authentication is performed using a virtual steering wheel in an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention, and FIG. 4 is this view A diagram illustrating an example of driving a vehicle using a virtual steering wheel in an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention.

Referring to the drawings, the remote control assistance system for an autonomous vehicle according to an embodiment of the present invention largely includes a virtual handle unit 100 and an autonomous vehicle 200 . If the autonomous driving level 5 level is implemented, the steering wheel inside the vehicle is also expected to disappear. Accordingly, in FIG. 1, the user's portable terminal is exemplified as the virtual handle unit 100 instead of the handle (vs). That is, the autonomous driving vehicle system having a virtual steering wheel according to an embodiment of the present invention is characterized in that the user directly intervenes in driving by taking over the control right of the autonomous driving vehicle 200 using a mobile phone terminal anywhere in the vehicle. do it with

At this time, referring to FIG. 2 , the user starts the user terminal constituting the virtual handle unit 100 to transfer the control right of the autonomous vehicle, and checks the health condition as shown in FIG. 3 to determine whether the condition is suitable for driving. After confirmation, the autonomous driving vehicle is controlled by rotating the virtual handle unit 100 as shown in FIG. 4 . In this case, the screen unit 140 of the virtual handle unit may output the alignment mark 141 to set a reference direction in which the user grips the first virtual handle unit 100 . It is preferable that such an environment is made by a passenger in the rear seat or without a steering wheel in the autonomous vehicle.

Hereinafter, an autonomous vehicle system according to an embodiment of the present invention will be described in more detail. 5 is a configuration diagram illustrating an autonomous driving vehicle having a virtual steering wheel according to an embodiment of the present invention, FIG. 6 is a configuration diagram illustrating the first sensing unit in FIG. 5 in more detail, and FIG. 7 is FIG. It is a block diagram showing the second sensing unit in more detail, and FIG. 8 is a block diagram showing the control unit in FIG. 5 in more detail.

Referring to the drawings, an autonomous driving vehicle according to an embodiment of the present invention includes a virtual handle unit 100 and an autonomous driving vehicle 200 as described above.

The virtual handle unit 100 is preferably formed of a user terminal, and transmits the first sensing unit 110 that detects the user's bio-signals or the user's manipulation signals and the user's manipulation signals to the autonomous vehicle 200 . It is made including a first communication unit. Sensing the user's bio-signal is to determine whether the user is a suitable person for driving or to determine the user's identity.

To this end, the first detection unit 110 is made of a biometric sensor such as a fingerprint recognition sensor or a camera to detect the user's biometric information to perform user authentication. In this case, of course, the virtual handle unit 100 may include a separate input unit 130 to input information for identifying an identity. In addition, the virtual handle unit 100 includes a separate screen unit 140 as shown in FIGS. 2 to 4 to output information on acquisition and conversion of control rights and a control environment.

In addition, the first detection unit 110 may include any one of the acceleration sensor 213, the gyro sensor 214, and the gravity sensor 215, by detecting the physical rotation value of the virtual handle part applied by the user. It is transmitted to the control unit 260 as a driving signal. On the other hand, it goes without saying that a separate acceleration/deceleration means or a stop means is output to the input unit 130 and selected to perform acceleration and deceleration of the vehicle.

The autonomous vehicle 200 includes a second communication unit 220 that receives a user operation signal received from the first communication unit, a control unit 260 that converts these signals, and a steering unit that receives the operation signal converted from the control unit 260 ( 230 , and a driving unit 240 for driving the autonomous vehicle according to a signal from the steering unit 230 . In addition, it may further include an output unit 250 for outputting driving environment related information.

Here, the steering unit 230 is a steering wheel of an actual autonomous driving vehicle or, in the case of an autonomous driving vehicle without a steering wheel, means a direction or speed applying means for controlling the driving unit 240 in place of a steering wheel.

The second sensing unit 210 includes at least one of a radar 211 , a lidar 212 , and a camera 213 .

The radar 211 refers to a device capable of moving unmanned while performing a radar detection operation by mounting an antenna in an autonomous vehicle. The radar 211 is not limited to performing only a radar detection operation using the provided antenna, and such as a plurality of sensors or complex sensors capable of sensing the surrounding environment such as the lidar 212 and the camera 213 . It can be implemented in the form The radar 211 senses the surrounding environment to generate surrounding sensing information, and based on the surrounding sensing information, determines the road, other vehicles, people, and inanimate objects that exist in the driving direction based on the surrounding sensing information.

The lidar 212 is a device that precisely draws a surrounding state by emitting a laser pulse, receiving the light reflected from the surrounding target object, and measuring the distance to the object. The lidar 212 emits a laser pulse to the ground surface and measures the return time by analyzing the spatial position of the reflection point to measure the topography, and the reflection and return time is different depending on the structure. A difficult three-dimensional model can be obtained. The lidar 212 is composed of a laser, a scanner, a receiver, and a positioning system, and the wavelength of the laser is 600-1000 nm. The scanner is the part that lets you quickly scan your surroundings to get information. The receiver is a part that detects the returning light, and the sensitivity of the receiver to the light is a major factor influencing the performance of the lidar. Essentially, the receiver detects photons and amplifies them. The positioning system performs a role of confirming the coordinates and direction of the location where the receiver is placed in order to implement a 3D image.

Meanwhile, the controller 260, which will be described later, maps the object captured by the camera 213 to the object determined by the radar 211 or the rider 212 to each other. That is, the controller 260 does not specify the object determined by the rider 212 or the radar 211 as the final object, but further maps the object with the camera thereto so that the output unit 250 outputs it. Accordingly, the user performs control while viewing object information, which is easily overlooked in the indirect control environment, in real time.

At this time, the control unit 260 uses the remote signal conversion module 261 that converts the rotation value detected by the first sensing unit 110 to transmit it to the steering unit 230 , and the user using the virtual handle unit 100 . Even while controlling the autonomous vehicle 200, a driving monitoring module 262 that compares the original autonomous driving control state with the user's control state, and an autonomous driving intervention module 263 that intervenes autonomous driving in the user's control state comprising at least one of

In more detail, the remote signal conversion module 261 is the user of the virtual handle unit in contrast to the left and right rotation of the steering unit (handle) of the autonomous vehicle based on the 180 degree rotation of the virtual handle unit 100 left and right. Convert the operation signal. In other words, in the case of holding an object such as a mobile phone with both hands and rotating it, it is not easy to rotate it more than 180 degrees left and right. Accordingly, in preparation for turning the left and right 180 degrees as a limit and similarly turning the actual steering wheel of the vehicle left and right 180 degrees, an operation signal necessary for driving the final vehicle is generated. However, if the vehicle handle is turned all the way to the left and right to completely change the direction of the vehicle, it is possible to rotate the virtual handle part 360 degrees with one hand so that the center becomes the axis. It is desirable to perform this after setting the parking mode.

On the other hand, in the case of holding and rotating the virtual handle unit 100 with both hands, initial alignment of the virtual handle unit 100 may be required. Therefore, it would be preferable to check the center indicator 141 and confirm that the rotation of the virtual handle unit 100 is not made in the correct position as in FIG. 4 described above.

In addition, the remote signal conversion module 261 is the measured value of the first detection unit 110 according to the change in the position of the arm or shaking of the hand independent of the rotation of the virtual handle when the user holds the virtual handle unit 100 is It plays a role in judging it as noise and removing it.

The driving monitoring module 262 compares and determines the original autonomous driving control state and the user's control state even while the user controls the autonomous driving vehicle 200 using the virtual handle unit 100 . This is because there may be cases where the judgment of an autonomous vehicle takes precedence in an unexpected situation, and in that case, there is room for autonomous driving to intervene again.

To this end, the autonomous driving intervention module 263 intervenes autonomous driving even in the user's control state. At the time when control by the user starts, a control non-adaptation problem occurs. It is judged that the user adapts to driving while helping, and the amount of autonomous driving intervention can be gradually reduced. In the worst case, it is of course possible to control so that no autonomous driving intervention takes place at all.

On the other hand, when it is determined by the aforementioned driving monitoring module 262 that the driving direction or speed controllable by the autonomous vehicle and the driving direction or speed controlled by the user are greater than the set value within the set time, autonomous driving The intervention module 263 may intervene to recover some or all of the control right of the autonomous vehicle. If the driving direction deviates significantly from the reference value in a very small time span, there is a risk of causing an accident, in which case the autonomous vehicle intervenes.

Hereinafter, a method of driving an autonomous vehicle according to an embodiment of the present invention will be described. 9 is a flowchart illustrating a driving method of an autonomous vehicle having a virtual steering wheel according to an embodiment of the present invention. Here, the content overlapping with the above will be omitted.

Referring to the drawings, in the method of operating an autonomous vehicle according to an embodiment of the present invention, the user requests to switch the control right of the autonomous vehicle (s10), the user is suitable for autonomous driving, but the user's health condition or the user's health condition In contrast to the step of confirming the identity (s20), the user starts to control the vehicle, but the entire control right of the autonomous vehicle is not transferred (s30), and the self-driving vehicle controls the same path as the user controls the vehicle and determining (s40) and a step (s50) of transferring the entire control right of the autonomous vehicle to the user.

Specifically, after step s50, when it is determined that there is a risk of collision according to manual control, the steps (s60 and s70) of the autonomous driving vehicle intervening in driving are further included. Accordingly, in terms of safety, the complementary harmonization of manual control and autonomous control of autonomous vehicles is further secured.

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms are used, these are only used in a general sense to easily explain the technical content of the present invention and help the understanding of the present invention. , it is not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

1000: autonomous vehicle system
100: virtual handle unit
110: first detection unit
120: first communication unit
130: input unit
140: screen unit
200: autonomous vehicle
210: second detection unit
211: radar
212: lidar
213: camera
220: second communication unit
230: steering unit
240: driving unit
250: output unit
260: control unit
261: remote signal conversion module
262: driving monitoring module
263: autonomous driving intervention module

Claims (10)

In the autonomous vehicle system,
a virtual handle unit including a first sensing unit for detecting a user's bio-signal or a user's manipulation signal, and a first communication unit for transmitting the user's manipulation signal; and
an autonomous vehicle comprising: a control unit that converts the user operation signal received from the first communication unit; a steering unit that receives the operation signal converted by the control unit;
including,
The autonomous driving vehicle system having a virtual handle, characterized in that the virtual handle unit further includes a screen unit on which the central state information of the virtual handle unit is output, and the user starts the control when the virtual handle unit belongs to the central state.
According to claim 1,
The self-driving vehicle system, characterized in that the first detection unit is composed of a biosensor or a camera to detect the user's biometric information to perform user authentication.
According to claim 1,
The self-driving vehicle system having a virtual handle, characterized in that the first detection unit is made of any one of an acceleration sensor, a gyro sensor, and a gravity sensor to detect a physical rotation value of the virtual handle unit applied by a user.
4. The method of claim 3,
The control unit includes a remote signal conversion module that converts the rotation value detected by the first sensing unit to transmit it to the steering unit, and even while the user controls the autonomous vehicle using the virtual handle unit, the original autonomous driving control state and the user's control An autonomous driving vehicle system having a virtual steering wheel, characterized in that it comprises at least one of a driving monitoring module for comparing a state and an autonomous driving intervention module for intervening autonomous driving in a user's control state.
5. The method of claim 4,
The remote signal conversion module is a virtual steering wheel, characterized in that it converts a user manipulation signal of the virtual handle unit in comparison with the left and right 180 degree rotation of the steering unit (handle) of the autonomous vehicle based on the 180 degree rotation of the virtual handle unit left and right An autonomous vehicle system comprising a.
delete 5. The method of claim 4,
The autonomous driving vehicle system having a virtual handle, characterized in that the autonomous driving intervention module increases the amount of autonomous driving intervention at the time when control by the user starts and gradually decreases the amount of autonomous driving intervention.
5. The method of claim 4,
The autonomous driving intervention module intervenes when the driving direction or speed determined by the driving monitoring module and the driving direction or speed controlled by the user are greater than a set value within a set time to recover part or all of the control right of the autonomous driving vehicle An autonomous vehicle system having a virtual steering wheel, characterized in that.
delete delete

Images